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ABSTRACT 


The goal of the Management System for Heterogeneous Networks (MSHN) is to 
provide a resource management system (RMS) to enable adaptive applications to use 
multiple sets of shared resources while accounting for dynamically changing priorities 
and environments. This RMS must be capable of providing each subscriber process with 
its required Quality of Service (which might include security considerations, deadlines, 
user priorities, and preferences) in a heterogeneous computing environment in which 
many processes are competing for shared resources. 

Applying this RMS technology to C4I modeling and simulation applications 
would enable on-scene Commanders to simulate complex elements of the decision 
process in order to optimize the use of forces and materiel. 

The objective of this thesis is to transparently intercept operating system calls 
made by a robust, C4I modeling application, the Extended Air Defense Simulation 
(EADSIM), in order to weigh the resources required against the confidence level of the 
outcomes obtained. Specifically, the goal is to determine resource usage required to run 
the application using both Monte Carlo simulation and deterministic simulation. MSHN 
needs this type of information to determine which version of an application to execute, in 


order to provide the best Quality of Service, while meeting operational deadlines. 
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EXECUTIVE SUMMARY 


The Defense Advanced Research Projects Agency (DARPA)-sponsored 
Management System for Heterogeneous Networks (MSHN) project is a sub-component 
of the DARPA QUORUM program. The goal of this project, as well as the program at 
large, is to provide a resource management system (RMS) to enable military computer 
applications to use multiple sets of shared resources while accounting for dynamically 
changing prionties and environments. MSHN’s RMS must be capable of providing each 
subscriber process with its required Quality of Service (which might include security 
considerations, deadlines, user priorities, and preferences) in a heterogeneous computing 
environment in which many processes are competing for shared resources. Rather than 
establishing this RMS as stand-alone software, the MSHN architecture is designed as an 
integrated system, integrated with and incorporating a variety of distributed system tools 
to reap the maximum benefits from available resources. 

In a military environment such an integrated system might mean offering the 
Commander the opportunity to select the most appropnate application, or version of an 
application, capable of executing within a specified time, at the proper security level, in 
order to deliver the best achievable answer within his stated time constraints. Applying 
this technology to robust C4I modeling and simulation applications would enable on- 
scene Commanders or mission planners to simulate complex elements of the decision 
process in order to optimize the use of forces and materiel. 

The need for robust C4I modeling and simulation in support of the tactical 
commander, the “Warfighter,” has been established in numerous Joint Warnor 
Interoperability Demonstrations, Fleet Battle Expennments, and Joint and service-specific 
exercises. However, to make the use of such models practical in a heterogeneous 
computing environment, the resource management concepts addressed by MSHN are 
needed. 

Key to the implementation of MSHN 1s the requirement for adaptive and 
adaptation aware applications. Adaptive applications exist in different versions capable 
of producing like results (though possibly offering varying degrees of QoS). MSHN 


would monitor the use of such adaptive applications and would be able to terminate one 
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version and start another, possibly from the beginning, if it perceived the user’s QoS 
requirements were not being met by the currently executing version. In a tactical 
environment, this means that the transition to improved QoS recommended by the MSHN 
RMS, if accepted by the decision-maker, would transparently enhance his or her mission 
effectiveness while remaining within given time constraints. 

This thesis presents the methodology of intercepting, or wrapping, system calls 
made by the Extended Air Defense Simulation (EADSIM), a robust C4I, air and missile 
warfare modeling application, in order to determine the resources required to execute the 
program on a stand-alone workstation. Having demonstrated the ability to measure the 
application’s resource usage without requiring access to source code, an experiment is 
described in which the resource usage 1s measured running the application in both Monte 
Carlo simulations and deterministic simulations. The outcomes obtained from running 
EADSIM in both deterministic and stochastic simulations are then weighed against each 
other. The overhead associated with the MSHN wrapper, a modified C library used to 
intercept system calls, is also measured and possible causes of this overhead are 
discussed. 

The research conducted for this thesis led to the realization that the MSHN 
wrapper needs to be expanded to collect finer grained information. Specifically, send{(), 
sendto( ), sendmsg( ), recv( ), recvfrom( ), recvmsg( ), select( ), and 
listen( ) system calls may need to be wrapped. Additionally, more information may 
be required from the current wrapper. Finally, the Binomial distnbution was used to help 
evaluate the trade-off between the fidelity of results from deterministic simulations and 
stochastic simulations. It was concluded that counter-missile events cannot be assumed 
to be independent in order to develop a measure of effectiveness to compare the 
deterministic version of EADSIM with the stochastic version. However, it was shown 
that the deterministic version of EADSIM offers valuable information while requiring 


approximately 1/20 the compute resources required by the stochastic version. 
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I. INTRODUCTION 


This thesis investigates the need for adaptive!, combat modeling; specifically as it 
relates to Command, Control, Communications, Computers and Intelligence (C4I). 
Further, it presents the results and analysis of resource measurement experiments run 
using the Extended Air Defense Simulation (EADSIM) program. The purpose of these 
experiments 1s to obtain data to facilitate the comparison of distributions of resources 
required to run the EADSIM program stochastically and the resources required to run it 


deterministically, weighing confidence levels of the results against the resources required. 


A. C4I MODELING AND SIMULATION IN A DISTRIBUTED 
ENVIRONMENT 


Future military missions will be highly diverse, including operations other 
than war (OOTW), often undertaken by joint, combined, or coalition 
forces. However, existing command support tools are not flexible enough 
to aid commanders in planning for such diverse and dynamic missions. 
[DESI98] 


The need for sophisticated C4I modeling and simulation? applications to support 
strategic planning has been accepted since the early part of the twentieth century when 
Frederick Lanchester developed his linear and square law equations during the First 
World War. However, the advent of “network-centric warfare,” the concept of 
metacomputing, advances in communication technologies (offering dramatic increases in 
data transfer rates), and the ability to take better advantage of distributed systems have 
presented the opportunity to apply these same models and simulations in an operational 


environment for use by the Warfighter in tactical planning. 


| Adaptive computer applications are those applications that, in order to support varying Quality of Service 
requirements, exist in different versions capable of producing like results. Adaptive applications should be 
idempotent, allowing them to be re-started in another version without corrupting any resources. 


2 Use of the terms “simulation” and “modeling” in this paper are in accordance with military common 
usage. The Department of Defense defines a “model” as: “A physical, mathematical, or otherwise logical 
representation of a system, entity, phenomenon, or process.” “Simulation” is defined as: “A method of 
implementing a model over time.””’ [MSMP95] 


The implementation of the Global Command and Control System (GCCS) in 
1996 heralded a major technological advance from its predecessor, the World Wide 
Military Command and Control System. GCCS software is a suite of applications 
intended to operate over a global network in support of the command and control mission 
of the United States and her coalition partners. As such it would seem to be a logical 
candidate as a venue for C4I modeling and simulation packages. The utility of having 
robust planning and decision tools available to the on-scene commander and his staff is 
obvious if (and only if) such tools can deliver data within very limited time constraints 
and with sufficient fidelity without robbing resources needed for the execution of such 
plans. 

Department of Defense Joint Publication 1 defines command and control as 


follows: 


The exercise of authority and direction by a properly designated 
commander over assigned forces in the accomplishment of the mission. 
Command and Control functions are performed through an arrangement 
for personnel, equipment, communications, facilities, and procedures 
which are employed by a commander in planning, directing, coordinating 
and controlling forces and operations in the accomplishment of the 
mission. [JOIN91] 


This definition was used by Frank Snyder [SNYD93] to break out three 
fundamental elements of command and control: 1) the command function; 2) the 
command and control process; and, 3) command, control and communications (C3) 
systems (C3 systems have more recently been expanded to include computers and 
intelligence, resulting in the popular use of “C4I systems” vice simply “C3 systems”). In 
addition to providing us a three sided prism through which we might diffuse the 
complicated spectrum of command and control, Snyder identified two essential variables 
that impinge on every aspect of command and control, particularly during combat: time 
and uncertainty. It is in an attempt to aid the decision-makers (the commanders) in 
successfully managing, or balancing, these two variables and their effects that modeling 
and simulation provide ready tools. This desire for a “balanced” response underscores 


the need for a means to manage computing resources (resource management) so as to 


bo 


ensure that high priority applications will execute with acceptable timeliness and level of 
confidence. 

The Lawson-Moose C4I Cycle (Figure 1. 1) provides a graphic depiction of the 
command and control decision process, commonly referred to as the “OODA” Loop, 
wherein the commander observes his environment (friendly and enemy forces, weather, 
terrain), processes the sensed information, compares his present state to the desired state, 


decides upon a course of action, and acts upon his/her decision. 


Lawson-Moose C’ Cycle 
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Figure 1. 1 Similarities between Lawson-Moose Cycle and the OODA Loop. After 
Ref. [PEND93] 


This cycle demonstrates each of the three elements of command and control noted 
earlier: the process (planning, directing, coordinating and controlling forces in the 
accomplishment of the mission); the command function (the exercise of authority and 
direction by a properly designated commander); and the system (the arrangement of 
personnel, equipment, communications, facilities and procedures which are employed by 
the commander). Each of these elements, within the context of the decision cycle, offers 
an opportunity for a model to assist the decision maker in improving his or her ability to 


make the right decision, thereby lessening uncertainty in a timely manner. 


In 1989 and 1990 the US Air Force Center for Studies and Analysis was using a 
raid simulation model, then-called C3ISIM, to study command, control, communications, 
and intelligence in various programmed scenarios [CASE91]. Developed for the US 
Army Space and Missile Defense Command (SMDC), C3ISIM (now known as 
EADSIM-Extended Air Defense Simulation) is an analytic, Monte Carlo3, and 
deterministic model of joint and combined force air and missile warfare, used for 
scenarios ranging from few-on-few engagements to theater-wide applications. It is a 
workstation hosted, system-level simulation used to assess the effectiveness of air and 
missile defense systems against a spectrum of air and surface-launched threats. EADSIM 
models fixed and rotary wing aircraft, tactical ballistic missiles, cruise missiles, a variety 
of bombs, high energy weapons, infrared and radar sensors, satellites, command and 
control structures, sensors and communications jammers, communications networks and 
devices, and fire support in a dynamic environment. 

As a result of the invasion of Kuwait, the Center for Studies and Analysis began 
adapting C3ISIM as an air planning support tool. A suite of powerful workstations was 
deployed to Riyadh along with a team of trained analysts. Populating the databases of 
C3ISIM was essential to successfully modeling the overall air campaign and its subsets. 
A key to the success of the effort, however, was that it was co-located with, and received 
extensive support from, joint military operators who were in the process of developing 
the actual Air Tasking Order (ATO). Despite the fact that these workstations were not 
connected to a wide area network (WAN) and were unable to take advantage of 
metacomputing or data mining, C3ISIM proved to be highly valuable in assessing 
potential sortie attrition, allowing planners to modify strike packages and compare results 
without risking lives and materiel. 

Because this early version of the model was extremely resource intensive (three 
hours of real time air operations required eight hours of simulation run-time), C3ISIM 
lost some of its usefulness as a timely mission planning tool following the 


commencement of Desert Storm. The high pace of operations and the rapidly changing 


3 While EADSIM is described by the program’s designer, Teledyne Brown Engineering, as a “Monte 
Carlo model”, it is perhaps more accurate to describe it as a discrete-event-simulation model that employs 
Monte Carlo methods to generate random variates. A detailed discussion of Monte Carlo and discrete- 
event simulation is available elsewhere. [KELT91] 


ATO caused analysts to turn their attention to “after-action” modeling. Post-mission 
analysis still involved the modeling of air missions, but was limited to specific target 
packages (vice the broad panorama of theater wide air operations). 

EADSIM modeling methodology was improved as time passed both in-theater 
and via remote technical support, but the real potential of this decision support tool was 
perhaps never realized. How much more useful could this model have been with reduced 
processing time (made possible by metacomputing), flexible resource scheduling 
(through efficient distributed system management), and an adaptable model capable of 


reconfiguring to meet quality of service requirements in a changing environment? 


Advanced modeling and simulation (M&S) may integrate a mix of 
computer simulations, actual warfighting systems, and weapon system 
simulators. The entities may be distributed geographically and connected 
through a high-speed network. Warriors at all levels will use M&S to 
challenge their military skills at tactical, operational, or strategic levels of 
war... [MSMP95] 


In bringing the model to the Warfighter in a distributed, heterogeneous 
environment, resource management tools (discussed in Chapter II) that can address 
specific Quality of Service parameters are needed. These parameters might include a 
network subscriber’s bandwidth allocation, application access priority (assigned by 
higher authority), preference for display of results (virtual reality, full motion video, high 
resolution images, graphical representations, text only, etc), preference for granularity 
and type of model chosen (stochastic, deterministic, time-stepped, event-driven), time 
constraints (perishability of material, timeline of pending military action), and 
subscriber’s security access (assigned by proper authority and subject to authentication 
and verification). Such parameters can then be used to allocate the resources necessary to 
provide the requested Quality of Service or to offer the subscriber the opportunity to 
select an adaptable application capable of reconfiguring to meet the subscriber’s needs in 
a changing environment. 

Modeling and simulation have a role to play in each of the three elements of 


command and control: the command function, the C4I process and the C4I systems. One 


logical venue for such modeling would be the Global Command and Control System 


(GCCS). 


GCCS is the central C2 system for achieving information 
superiority in the Joint Vision 2010. It is an integrated, reliable and secure 
‘command and control system linking the National Command Authority 
(NCA) to the Commander in Chiefs (CinC) of the major commands down 
to the Joint Task Force (JTF) and Component Commanders. As the top 
level infrastructure for automated support to C4I operations worldwide, it 
is to provide a seamless battlespace awareness by exchanging data, 
imagery, intelligence, status of forces, and planning information... GCCS 
employs (sic) client/server architecture using commercial software and 
hardware and open systems standards. Currently, GCCS integrates SUN, 
HP, and PC products and operating systems with ORACLE and SyBase 
distributed relational database support. [ANTH98] 


In order to take advantage of the distributed resources and metacomputing 
necessary to incorporate robust modeling applications into GCCS (or to make them 
available by commercial WEB browser over a secure network), it 1s essential that models 
be adaptable and capable of working in a distributed system. It is that adaptability that 
will make the use of such models practical as a tactical decision aid for the Warfighter in 


a distributed, heterogeneous computing environment. 


B. SCOPE OF THIS THESIS 


In order to support adaptive and adaptation aware applications (defined fully in 
Chapter II, Section C) the Management System for Heterogeneous Networks (MSHN) 
architecture includes a Client Library to intercept system calls and measure the resources 
required to run an application*. The Client Library is transparently linked with each 
application to allow for the gathering and processing of useful data from system calls. 
This process should be transparent to the user’s application, should require minimal 


overhead, and should be accomplished without the need for application source code. As 


4 MSHN employs a method of intercepting an application’s request for hardware before it reaches the 
operating system by wrapping the application within a composite library, referred to as the MSHN wrapper. 
The MSHN wrapper intercepts a system call, adds pre- and/or post-processing functionality for measuring 
resource uSage, and returns the value of the system call to the requesting application. 


shall be discussed in Chapter II, the data collected from these wrappers are used by the 
components of MSHN to establish resource requirements, resource status, and 
Optimization criteria; to make scheduling decisions; and to monitor application 
performance in order to ensure adequate quality of service. A wrapper for this purpose 
was designed and demonstrated in a precursor thesis [SCHN98] and was proven capable 
of gathering an application’s resource usage data without the need for that application’s 
source code and without adding excessive overhead. 

The objective of this research was to wrap a robust, C4I modeling application, 
representative of complex modeling applications currently in use by the Department of 
Defense, in order to determine the resources required to execute the application on a 
stand-alone workstation. Specifically, the goal was to determine resource usage required 
to run the application using both Monte Carlo simulation and deterministic simulation. 
The resources required would then be weighed against the confidence level of the 
outcomes obtained. MSHN needs this type of information to determine which version of 
an application to execute, in order to provide the best Quality of Service, while meeting 


operational deadlines. 


Cc: MAJOR CONTRIBUTIONS OF THIS THESIS 


The need for robust C4I modeling and simulation in support of the tactical 
commander, the “Warfighter,” has been established in numerous Joint Warrior 
Interoperability Demonstrations, Fleet Battle Experiments, and Joint and service specific 
exercises (discussed in Chapter II B 2). However, to make the use of such models 
practical in a heterogeneous computing environment, the resource management concepts 
addressed by MSHN are needed. Key to the implementation of MSHN is the 
requirement for adaptive and adaptation aware models that offer different versions of an 
application to meet varying Quality of Service demands. Before scheduling algorithms 
can be tuned to support such applications, more needs to be known about the actual 
resource requirements of such models. This thesis provides the first such data gathered 
on a complex, contemporary, C4I/air defense model currently in use throughout the DoD, 


with conclusions drawn regarding the trade-offs of computing resources and confidence 


in simulation outcomes. The data from this thesis will be used in follow-on research 
intended to determine the distributions of these resources. Once the distributions have 
been determined, resource data collected from EADSIM simulations will be used to 


develop a MSHN application emulator (described in Chapter II). 


D. ORGANIZATION 


This thesis is organized as follows: Chapter II addresses the need for distributed 
systems in order to take advantage of metacomputing in a heterogeneous computing 
environment. It discusses three tools designed to support distributed systems, and the 
role of MSHN in integrating such tools to support adaptive and adaptation aware 
applications. Additionally, it discusses the most closely related work of which we are 
aware: Armstrong’s collection and analysis of large grained resource usage by 
Numerical Aerodynamic Simulation (NAS) benchmarks [ARMS97, HENS99]. Chapter 
III describes EADSIM’s technical and operational architecture. Chapter IV explains the 
concept of the MSHN wrapper, and the specifics of wrapping EADSIM run-time 
executables. Chapter V provides an explanation of why EADSIM was selected to 
represent an adaptive application. The EADSIM scenario used in the experiment is 
described in detail, as is the measure of effectiveness chosen to weigh simulation 
outcomes against resource requirements. Chapter VI provides a description of the 
experiment including its methodology, the computing environment in which it was 
conducted, the resource measurement data gathered from running EADSIM in both 
Monte Carlo and deterministic configurations and analysis of the resulting outcomes 
from those runs. Chapter VI also offers possible explanations of the overhead added by 
the MSHN wrapper and an evaluation of the measure of effectiveness described in 
Chapter V. The final chapter provides conclusions and suggestions for future research 


and its application to C4I. 


Il. THE ROLE OF MSHN AND RELATED WORK 


We should expect to participate in a broad range of deterrent, conflict 
prevention, and peacetime activities. Further, our history, strategy, and 
recent experience suggest that we will usually work in concert with our 
friends and allies in almost all operations...Improvements in information 
and systems integration technologies will also significantly impact future 
military operations.... [JOVI95] 


This chapter places MSHN into perspective by describing related middleware 
standards and projects and explaining why such middleware is needed in what has 
become known as “network-centric warfare.” Additionally, it describes the previous 
research most closely related to this thesis, Armstrong’s work [ARMS97] with NAS 
benchmarks. 


A. THE NEED FOR DISTRIBUTED SYSTEMS 


While the geographic setting, venue, scope, and magnitude of future military 
operations is uncertain, they will likely involve non-co-located Commanders-in-Chief 
and/or Component Commanders, each facing unique work-space challenges, forced to 
function in a heterogeneous computing environment with insufficient on-site computing 
resources and constrained bandwidth. Such is the nature of warfare, low intensity 
conflict, and operations other than war in the information age. It was, perhaps, this 
environment that VADM Cebrowski [CEBR98] had in mind when he coined the term 
“network-centric warfare.” With limited on-site resources and geographically disparate 
command and control elements, full advantage must be taken of collaborative planning 
tools, common databases, and powerful off-site computing assets. This can only be 
accomplished through the use of distributed systems. 

A distributed system is a set of computers, connected by at least one network, that 
do not share memory or acommon clock. The goal of a distributed system is to cause a 
set of computers to appear to the user as a single, powerful virtual machine. This is true 


whether accomplished through the use of a distributed operating system (allocates fine- 


grained resources of the virtual machine to application processes), a resource 
management system (allocates single machines or groups of machines within the virtual 
machine to application processes, but allows each machine to run its native operating 
system), or a distributed computing environment (provides paradigm libraries or 
programming language support to facilitate the sharing of available resources). There 
are five primary advantages offered by distributed systems: resource sharing (hardware 
and software can be shared among computers); enhanced performance (higher throughput 
and speed through increased concurrency); improved reliability (through replication of 
data files and services, distributed systems can be made more fault tolerant); improved 
availability (Some components may fail without affecting the overall performance of the 
system); and modular expandability (hardware and software can be added without 
adversely impacting existing resources). Due to the fact that network latency 
(unbounded message passing time) and heterogeneous architectures make a global clock 
and completely consistent shared memory infeasible, it is impractical for a distributed 
system to have a coherent, total view of the global state. Therefore, synchronization and 
consistency of state present challenges to any distributed system. This means that each of 
the advantages cited above has accompanying pitfalls that must be carefully considered 
when designing the distributed system and selecting global state algorithms to detect 
stable properties (i.e., process termination, deadlock, garbage collection). The need for 
consistent global states is no where more apparent than in dealing with discrete event 
simulation models that incorporate both event-driven and time-stepped events. EADSIM 


is such a model. [SING94] 


B. THREE MIDDLEWARE STANDARDS AND PROJECTS FOR 
DISTRIBUTED SYSTEMS 


In the last few years, clusters of LAN°- or WAN- connected systems have 
become a reasonable and cost effective alternative to the use of expensive, 
dedicated monolithic high-performance systems. The notion of a 
metacomputer has been coined, denoting a ‘network of heterogeneous, 
computational resources linked by software in such a way that they can be 
used as easily as a personal computer’. [BRUN97] 


> Local Area Network 
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MSHN researchers do not foresee the results of their resource management 
system (RMS) research as a cumbersome piece of software requiring separate installation 
and maintenance. Rather, the outcome of MSHN’s research may be packaged as a 
middleware-level standard. An RMS packaged in this way would eliminate the need for 
separate installation and could be consolidated with the services that distributed 
applications use most often. This section discusses middleware standards and projects of 
interest to MSHN investigators. 

Middleware is software that sits between applications and lower level 
communication protocols and operating systems. The purpose of middleware is to 
provide a variety of services (i.e., naming, security, load balancing, resource brokering, 
and communication) to applications being executed in a distributed, and perhaps 
heterogeneous, computing environment. Through the use of middleware, applications 
written in different languages, executing via different operating systems, might access 
common services and achieve interoperation. 

The Object Management Group (OMG) is a consortium of more than 800 
companies, whose goal is to specify a middleware architecture consisting of an object 
request broker, common object services and vertical application domains. While it is not 
OMG’s intention that this architecture specification control the way in which middleware 
technology is implemented, the consortium is interested in ensuring the interoperation of 
implementations currently being developed. Common Object Request Broker 
Architecture is the evolving middleware standard being nurtured by OMG. [DOLG99] 

This section discusses CORBA and two other middleware tools, and how they 


relate to MSHN’s research. 


1. CORBA 


In an organization as vast and complex as the US military, it is unreasonable to 
believe that computing environments in the future will be anything but heterogeneous. 
Technological progress, the cumbersome DoD acquisition system (different for each 


service), varying needs and working environments, a variety of networks and 


Ih. 


accompanying protocols, and the unfeasibility/unwillingness to simply discard proven 
legacy systems and architectures assure that any large internet/intranet must be built from 
heterogeneous hardware, software and operating systems. While heterogeneity in itself is 
not negative and may, in fact, be viewed as an asset to be leveraged, it does present 
challenges to software developers and contractors desiring to take advantage of 
heterogeneous networked systems. Heterogeneity creates the need for middleware that 
can enable applications to share objects, functions and types without causing extensive 
software re-work for developers, or complex work-arounds for users. 

The Object Management Group (OMG) was formed in 1989 to develop, adopt, 
and promote standards for the development and deployment of applications in distributed 
heterogeneous environments. Since that time, the OMG has grown to be the largest 
software consortium in the world, and has developed the Object Management 
Architecture (OMA). The OMA consists of an Object Model and a Reference Model. 
The Object Model defines how objects can be described, and the Reference Model deals 
with interactions between those objects. In the Object Model clients issue requests for 
services to objects (much like a remote procedure call (RPC)). The implementations of 
these objects are hidden from the client. A key component of the Reference Model is the 
Object Request Broker (ORB), which facilitates communication between clients and 
objects (Figure 2. 1). Common Object Request Broker Architecture (CORBA) is the 
specification developed by the OMG that details the interfaces and characteristics of the 
ORB. In CORBA the terms “client” and “server” are roles that are filled on a case by 
case basis. A client for one request might be server for another ,[VINO96]. 

In CORBA, an application consists of one or more objects that may reside on the same or 
different platforms. An object provides service(s) that can be “requested” by a client 
(Figure 2. 1). Clients obtain services from an object by making “requests” (RPC-style 
requests, similar to a SEND operation, or by separate, deferred-synchronous, 
SEND/RECEIVE operations) that consist of an operation, the name of the object that will 
respond, zero or more parameters, and an optional request context. The object may or 
may not return results to a client, and will return an exception if an abnormal condition 
occurs. Code that is executed to perform a service is called a method. That is, a method 


defines the implementation details of an operation. Object Adapters are the run-time 


components of CORBA that sit between the ORB and the Object Implementations. 
Object Implementations may be written in a variety of languages and may exist in a 
variety of forms. With additional Object Adapters it is possible to support any style of 
object implementation. An Implementation Repository is used by the Object Adapter to 
provide run-time access to information about all currently available objects. [DUMA98] 
Methods can be invoked statically or dynamically. In Static Invocation, a client’s 
request 1s made via interface definition language (IDL) “stubs” on the client side, and the 
response is handled by IDL “skeletons” on the object side. The stubs and skeletons 
interface with the CORBA ORB. In Static Invocation, the IDL Client Stub converts data 
from the client’s local data representation (type) to the Common Data Representation 
(CDR), which is platform and language independent. On the object’s platform, the 
Object Skeleton executes the reverse operation. In Dynamic Invocation, requests are 
made via Dynamic Invocation Interface (which allows the IDL stubs to be replaced by 
separate SEND and RECEIVE operations). With Dynamic Invocation the developer is 
afforded more flexibility. In Dynamic Invocation, the Dynamic Skeleton Interface (DSI) 
may take the place of the Static Invocation Object Skeleton to accomplish data 


conversion at run time. 
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Figure 2. 1 Common Object Request Broker Architecture. From Ref.[Vino96] 


CORBA supports two types of (method) invocation semantics: synchronous 


invocation and asynchronous invocation. Synchronous invocation is blocking. The 


is 


client will invoke the method and block until it receives a response from the server 
(object implementation). With blocking primitives, the user buffer can be reused as soon 
as control 1s returned to the user program (when the message has been sent or an 
acknowledgement has been received). The RECEIVE primitive does not return control 
to the object executing it until a message has been copied into the buffer provided by that 
object. With synchronous primitives, a request (SEND) primitive is blocked until a 
corresponding RECEIVE primitive is executed at the receiving computer. 

Asynchronous invocation 1s non-blocking. The client will invoke a method, 
continue its computation, and collect results as they arrive. With non-blocking 
primitives, the SEND primitive returns control to the requesting program as soon as the 
message 1s copied from the user buffer to the keel buffer. The corresponding object 
that executes the RECEIVE primitive signals its intention to receive a message, provides 
a buffer into which the message will be copied and continues to execute. In CORBA the 
client can also make “‘one way” requests, continuing to execute while the object processes 
the request. [DUMA98], [SING94] 

Through the IDL stubs, a client can use RPC-style semantics (synchronous), or by 
using Dynamic Invocation Interface (DII) a client can use SEND/RECEIVE semantics. 
Using DII allows a client to directly access the underlying request mechanisms provided 
by the ORB. Applications use DII to dynamically issue requests to objects without 
requiring IDL stubs to be linked in. The DII allows clients to make non-blocking 
“deferred synchronous” (separate SEND and RECEIVE operations) and one way (SEND 
only) calls. [SCHM99] 


The Global Command and Control System—Leading Edge Services 
(GCCS-LES) is a platform for the transfer of advanced applications from 
the Research and Development community to the Defense Information 
Services Agency’s (DISA) GCCS_ service...The GCCS-LES 1s 
converging the architectures in the Modeling and Simulation (M&S) 
communities with Command, Control, Communications, Computer and 
Intelligence (C4]) architectures. [TBMC97] 


DISA recognized the importance of CORBA middleware in realizing the potential 
of a heterogeneous, distributed system and incorporated it in the GCCS-LES. In fact, 


CORBA is now part of the Defense Information Infrastructure Common Operating 
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Environment (DIJ COE) standard web browser, and is finding increasing use in DoD C4] 
applications. The utility of CORBA lies in its ability to integrate diverse applications 
across a variety of networks and network protocols. CORBA’s language independent 
IDL’s allow interfaces to be used from a variety of programming languages, including 
COBOL, C, C++, Ada, Smalltalk, Perl and Java. CORBA-based applications are 
independent of network protocols so they may be run in a distributed system over a 
diverse network. These attributes ensure CORBA’s usefulness in a heterogeneous 


command and control computing environment. 


1 COMPASS 


In 1994 the DoD Modeling and Simulation Office began sponsoring the US 
Navy’s Common Operational Modeling, Planning and Simulation Strategy (COMPASS) 
Project based at SPAWAR Systems Center, San Diego. 


The goals of the COMPASS project were to: prototype the use of a 
common messaging environment to allow M&sS services to better support 
C4I process; demonstrate the operational benefits to joint Warfighters of 
DCP tools to support M&S services for C4I/MP® systems; (sic) facilitate 
interoperability of M&S with C4I systems. [COMP99] 


COMPASS sought a means for geographically separated Commanders, 
operational planners, and analysts to collaborate in a virtual environment as effortlessly 
as they would if they were physically co-located (Figure 2. 2). In order to do this, they 
would need distributed collaborative planning (DCP) tools that supported legacy systems. 
These systems include modeling and simulation, C4I and mission planning systems that 
were already in place. This collaboration would have to be accomplished in a distributed, 
heterogeneous environment accessed by joint services and coalition partners. These 
decision makers and mission planners would have access to a range of bandwidths, 
possibly limited local computing resources, definite time constraints, and most 


importantly the need to share a common image of the battlefield. The COMPASS 


© Command, Control, Communications, Computers, and Intelligence/Mission Planning 


15 


project decided to pursue their goals through a combination of commercial off the shelf 
(COTS) and government off the shelf (GOTS) collaborative services integrated by 


middleware (Figure 2. 3). 
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Figure 2.2 SPAWAR Presentation Slide of COMPASS. After Ref.[MCSW99] 
. COMPASS Software Architecture 


¢! COMPASS Servers 


F Track .@3mdabon 3 


| 

‘ $ 

| 4 Cumoovie Rate T° 
| | Hedem Qurwr 













COMPASS-Capabie GCCS34 










COMPASS Client | raries ) 


Mista % 













GCTSICOTS Conferencing Apps 






FCPAP Network 









| COMPASS-Capable Legacy Modeling, Pienning, or Simulation System 
COMPASS Gient t ibrasies 


Megan fetmariad 





Legacy Syston Baseline 


Ee’. = ee as 
‘ « 








lc tn 






I 


| COMPASS SST 










GOTSICOTS Conterencing Apps 
— oe 


i 
[7 corsicors 
{ 
MOONE Mort i MBONE Video ii wrienaart Cert } 
t 





i j Legacy Syston Sw 


B 





Figure 2. 3 COMPASS Software Architecture. From Ref. [MCSW98] 
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COMPASS middleware is a peer to peer, client-server architecture that offers core 
Services to COMPASS-enabled systems accessing COMPASS servers through an 
Application Program Interface (API). COTS core services include: Whiteboard (enables 
sharing of graphics and images that can be cut and pasted, with tools for drawing and 
typing); Chat (allows low bandwidth exchange of typed information between multiple 
stations); Visual-Audio Teleconferencing/Video Interactive Conferencing (VAT/VIC) 
(VAT provides for video/audio teleconferencing via existing C4I/M&S applications, 
while VIC allows the sharing of video-based M&S products); and Collaborative Virtual 
Workspace (CVW) (establishes a “virtual DCP Conference center’ in which participants 
can access collaborative planning sessions and COMPASS services through centralized 
and well organized virtual venues). GOTS core services consist of Session Management 
(provides the means to create, join or monitor collaborative sessions); Shared Overlay 
Management (enables the sharing of a variety of geo-registered overlays); Composite 
Mission Preview (allows planners to pre-view a complete, animated mission plan); 
Simulated Mission Rehearsal (permits the viewing of Distributed Interactive Simulation 
(DIS)-capable models and simulations); and Track Data Base Management Server (Track 
Server) (tracks from GCCS can be shared and updated with COMPASS-capable and non- 
GCCS COMPASS capable workstations). This suite of DCP services provides the 
capability of integrating a variety of modeling and simulation, mission planning, and C4] 
systems into a common view during the planning process by ensuring that users are 
working with displays that show common tracks and are updated from a shared data base 
manager with geo-registered overlays (routes, weapons effects, etc.). Through a session 
management facility, the COMPASS architecture provides start-up, recovery (re- 
establishment of system state following abnormal termination), and backup Services. 
There are currently twelve M&S systems (including EADSIM) and eight C4I/MP 
systems that are considered COMPASS-enabled, having incorporated the middleware 


code necessary to access COMPASS services (Figure 2. 4). 
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Figure 2.4 COMPASS Capable Modeling and Simulation and C4I Systems. After 
Ref. [MCS W998] 


Since 1995 COMPASS has participated in more than twenty Joint and Service 
sponsored demonstrations and exercises, including four Joint Warrior Interoperability 
Demonstrations (JWID 1995, 1996, 1997 and 1998) and two Fleet Battle Experiments 
(FBE C, D). In each of these, COMPASS used metacomputing, reach-back and anchor 
desk concepts to support the Warfighter with robust distributed collaborative planning (in 
turn supported by modeling and simulation). Through these extensive operations 
COMPASS has repeatedly demonstrated not only the usefulness of COMPASS services, 
but the potential contribution of tactical modeling and simulation as an integral part of the 
C4lI process. COMPASS is now transitioning from an R&D program to an operational 


implementation. 


2: ENSEMBLE 


The Ensemble system, developed at Cornell University, is a network architecture 


designed to support network and application adaptation to changes in environment, users, 
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or application requirements. It is based on the use of protocol stacks to facilitate 
enhanced communications between applications (groups and group members) by 
supporting data replication, collaboration, or coordination. Ensemble is also capable of 
controlling or managing a network application in a manner transparent to the application 
developer. Elements of an application in a distributed environment include the users, the 
components of the application, the network, and the communication infrastructure and 
protocol. Adaptation can be achieved by reconfiguring any of these elements in response 
to a change in environment or requirements (Figure 2. 5). This reconfiguration might 
involve adding or deleting users from a group, changes in the communication 
infrastructure (bandwidth, protocol, security, etc.), or changes in the application 
(resources needed) itself. All of these responses to a changing environment require 
careful monitoring, control, and synchronization of the system state. Ensemble 
addresses this need through the use of a layered protocol architecture. 

In Ensemble, micro-protocol modules are used to meet the communications needs 
of an application. Sliding windows protocols, fragmentation and re-assembly, flow 
control, encryption, group membership and message ordering may be stacked in a variety 
of ways to address these needs. Each configuration of an application and environment is 
served by a stack of these protocols. Assumptions are made each time a configuration is 
initiated. These assumptions are then monitored by “detectors” (which can be micro- 
protocols themselves) that sense changes in the environment (actually violations in the 
assumptions of the currently monitored configuration) and provide the protocol stack 
with enough information to form the basis of new assumptions in the event of a 
reconfiguration. The determination of when to reconfigure is based on crossing a preset 


threshold of violations for the given assumptions. 
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Figure 2.5 When the Environment Changes, the Opportunity to Adapt is Passed 
Up the Protocol Stack Until a Layer Adapts or the Application is Notified 
(Requiring Reconfiguration). After Ref.[HAYD97] 


In the Ensemble layered architecture, the lowest layer protocols attempt to adapt 
to environmental changes first. If they are unable to do so, they notify the layer directly 
above them. This continues until either one of the protocols on the stack is able to adapt, 
or the application itself is notified. Once notified of the changed environment, the 
application must decide whether, or how, to reconfigure (Figure 2. 5). A reconfiguration, 
or “adaptation,” might involve the application adjusting its Quality of Service demands, 
dynamically modifying its interface with other applications, increasing or reducing its 
security requirements, increasing or decreasing its bandwidth requirements, or possibly 
even ceasing service to one or all application users. Each reconfiguration will result in a 
new protocol stack to support it (each “instantiation” of a Protocol Stack has a unique 
Protocol Stack Instance Identifier, PSI-ID). 

Any reconfiguration in a distributed environment must address consistency of 
state which in turn 1s likely to involve finalization, synchronization and initialization (or 
start-up) of a new state. In Ensemble the mechanism that creates and installs a new stack 
across a set of application users when a reconfiguration occurs is called a Protocol 
System Protocol (PSP). The PSP, itself a stack of protocols within the application’s 
stack, handles the finalization of the micro-protocols in the old stack, creates a new stack 


(assigning it anew PSI-ID), and starts the new protocol stack. 
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In order to reconfigure, the PSP selects the user with the lowest network address 
as a kind of “reconfiguration coordinator.” The chosen coordinator (the user that has 
been selected) generates the new PSJ-ID and broadcasts a finalize message to each user. 
This message contains a description of the new stack, the list of users, and the new PSI- 
ID. Each recipient builds the new stack of micro-protocols, registers the PSJ-ID and 
passes a finalize event to the top layer of the old stack. When each layer of the old stack 
is ready to stop sending messages, the finalize event is passed down to the layer below. 
When the finalize event has been processed by the bottom most micro-protocol, a 
finalize--acknowledge message is returned to the coordinator. When the coordinator 
collects all finalize--acknowledge messages, it broadcasts a start message to the new 
stack. Each recipient in turn passes a start event to the bottom most layer of the new 
stack, discarding the old stack. The start event is passed up the new stack and discarded 
by the top layer. For fault tolerance, in the event a user is unreachable or there is a 
network failure, the coordinator will broadcast a new finalize message (containing the list 
of all users in the group). The coordinator awaits finalize messages from users of both 
the old and new stack. If the coordinator fails, the user with the next lowest network 
address broadcasts the finalize message. PSP will avoid deadlock; if necessary a user is 
capable of installing a stack with a single element and resuming operation. A MERGE 
micro-protocol may be used to locate concurrent PSI’s for the same application, and to 
merge them. [HAYD97] 

An increased dependence on resource sharing has created the need for more 
reliable means of managing and controlling applications in a dynamic, distributed 
environment. Ensemble addresses this need by providing layers of micro-protocols that 
pass events up and down a stack, responding only to messages of interest to their own 
layer (Figure 2. 6). A system of checks and balances is used wherein assumptions are 
determined when the stack is formed, and violation detection monitors watch for changes 
in the environment that could signal the need for reconfiguration (adaptation). 
Synchronization, finalization and consistent state reconfiguration are ensured (by 
implementing the Process System Protocol). The Ensemble system is one approach to 
making adaptive applications in a distributed system a reality. The potential exists for a 


system like Ensemble to incorporate middleware like CORBA or COMPASS and offer 
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protocols that would enable adaptive modeling and simulation applications to span the 


most diverse, heterogeneous C4] network. 
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Figure 2.6 The Ensemble Architecture. After Ref.[HAYD97] 


cC. THE ROLE OF MSHN 


Network intensive computing places unusual stress on conventional 
computer system management and operation practice... Because 
significant remote computing and storage resources may be necessary, 
standardized services for resource allocation and usage accounting are 
important. Other important issues are enforcing the proper use of network 
resources, determining the scale and quality of service available, and 
establishing priorities among the users and uses. Mechanisms are needed 
to address these issues automatically and dynamically. [NSTB96] 


The Defense Advanced Research Projects Agency (DARPA)-sponsored 
Management System for Heterogeneous Networks (MSHN) project is a sub-component 
of the DARPA QUORUM program. The goal of this project, as well as the program at 
large, is to provide a resource management system (RMS) to enable both C4I applications 
and planning applications (indeed, any adaptive application) to use multiple sets of 
shared resources while accounting for dynamically changing priorities and environments. 
This RMS must be capable of providing each subscriber process with its required Quality 
of Service (which might include security considerations, deadlines, user priorities, and 
preferences) in a heterogeneous computing environment in which many processes are 


competing for shared resources. Rather than establishing this RMS as stand-alone 


22 


software, the MSHN architecture is designed as an integrated system, integrated with and 
incorporating a variety of distributed system tools (i.e. CORBA, ENSEMBLE, 
COMPASS, etc) to reap the maximum benefits from available resources. 

In a military environment such an integrated system might mean offering the 
Commander the opportunity to select the most appropriate application, or version of an 
application, capable of executing within a specified time, at the proper security level, in 
order to deliver the best achievable answer within his stated time constraints. 
Specifically, applying this technology to robust C4I modeling and simulation applications 
would enable on-scene Commanders to simulate complex elements of the decision 
process in order to optimize the use of forces and materiel. The key to this 
implementation however, is the development of adaptive and adaptation-aware models 
and decision support applications that can be scaled to fit user-defined Quality of Service 
(QoS) parameters. 

We recall that an RMS allows the user to view a set of computers as a single 
powerful virtual machine and does not provide micro-management of resources. Rather, 
as stated earlier, an RMS allocates single machines or groups of machines within the 
virtual machine to application processes, allowing each machine to run its native 
operating system. MSHN acknowledges that in a heterogeneous computing environment 
such as exists in the DoD, there will be little chance of an overarching distributed 
Operating system controlling all resources such as network routing, protocols, and server 
memory allocation. It is MSHN’s goal to simultaneously support multiple processes, 
each with a variety of QoS requirements while dynamically combining these 
requirements into a consolidated optimization criterion, then using this criterion to 
optimize the application of resources to these processes. In this way, MSHN will 
provide support for adaptive and adaptation-aware applications. 

MSHN’s use of the terms “adaptive” application and “adaptation-aware” 
application is best explained in terms of Quality of Service support. Adaptive 
applications exist in different versions capable of producing like results (though possibly 
offering varying degrees of Quality of Service). For instance, an adaptive application 
might have one version that executes on Linux, while others execute on Solaris, Unix or 


NT. MSHN would monitor the use of such adaptive applications and would be able to 
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terminate one version and start another version from the beginning if it perceived the 
user’s QoS requirements were not being met by the currently executing version. It is 
clear from this example then, that adaptive applications must be idempotent to allow an 
application to be started (in another version) without corrupting any resources (such as a 
database). An adaptation-aware application 1s similar to an adaptive application except 
that when terminated, it is not necessary to restart the new version from the beginning. 
Information about a previous state taken from the older version (that 1s about to be 
terminated) can be safely used in the newly started version. MSHN would thus monitor 
an adaptation-aware application as well as resources available elsewhere in the 
distributed system, and, upon finding resources to run a version of the application that 
could provide better QoS to the user, start the preferred version from the current state of 
the old version (then terminate the old version). Adaptation aware models could allow 
the user to “upgrade” to improved quality of service when resources become available 
without a costly loss of time or data. In a tactical environment, this means the transition 
to improved QoS would be transparent to the decision-maker, with no ill effect on 


mission planning or timing and potentially considerable positive effect. 


1. MSHN Architecture 


The primary elements of the MSHN architecture are the Client Library (CL), the 
Resource Requirements Database (RRD), the Resource Status Server (RSS), the 
Scheduling Advisor (SA), the MSHN Daemon (MD) and the MSHN Application 
Emulator (AE). The gateway of this architecture is the Client Library (Figure 2. 7). 
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Figure 2.7 MSHN’s Conceptual Architecture. From Ref.{[HENS99] 


To avoid requiring a MSHN user to explicitly log into the MSHN RMS, it was necessary 
to intercept all calls to system libraries that would initiate new processes and to divert 
these calls to the MSHN CL. Upon receipt of a request to launch an application, the CL 
checks to see whether the application is on a list of applications that need not be managed 
by MSHN. If it is, then the request is passed to a local operating system for execution. 
If, on the other hand, it is an application that does require MSHN management, the CL 
invokes a scheduling request on the SA, sending along the QoS requirements defined by 
the user. The SA then consults the RRD, to determine what resources are required based 
on the user’s QoS parameters. Afterwards, the SA polls the RSS to see what resources 
are currently available, and, based on this information and the optimization criteria, the 
SA decides where the process should execute and returns this choice of location to the 
CL. Based on input from the SA the CL then requests the Daemon located on the 
selected remote machine to execute the application (this will be accomplished through the 
use of CORBA, though not directly via the CORBA ORB). After the application has 
been launched, the CL is capable of responding to callbacks from the SA by requesting 
Daemons on other remote machines to launch preferred versions of adaptive or 
adaptation-aware applications. Daemons are also used to start the AE in order to 
simulate the running of applications without the accompanying overhead (to produce 


predictive statistics regarding resource requirements), or to monitor the status of 
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resources not being used by MSHN applications. Key to the implementation of this 
architecture is the “wrapping” of user applications by the CL. 

In order to provide MSHN services transparently to the user, the CL must 
intercept, pre-process and, when required, divert system library calls from user 
applications. This includes calls to exec, all socket calls, and calls to open, close, 
read, and write. By pre- and post-processing these intercepted calls to the operating 
system, the CL can determine the resources used by an application while it is running, 
store this information in the RRD, update the RSS as a process continues to execute, and 
call upon the SA to use current and historical data to determine where it is best to run the 
application in order to support its QoS requirements. It is the wrapping of applications 
that enables the CL to drive the functionality of all MSHN components. It is a goal of 
MSHN that this wrapping will eventually be accomplished by means completely 
transparent to the user. Although it is anticipated that this will be accomplished through 
the use of CORBA middleware, MSHN has already proven the ability [SCHN98] to wrap 
applications without requiring source code (object files however are still required). A 


more detailed explanation of MSHN can be found elsewhere [HENS99]. 


D. RELATED WORK 


MSHN evolved from a scheduling framework called SmartNet. The goal of the 
SmartNet project was to improve performance by applying sophisticated scheduling 
capabilities to a set of compute-intensive jobs, each of which may require multiple 
processes, onto a set of heterogeneous computers. SmartNet predicted the expected run- 
time of a job on a particular machine based upon the wall-clock time required by 
previous executions of the job. The wall-clock time was partitioned according to 
compute characteristics [KIDD99]. A scheduling module leveraged the heterogeneity of a 
set of jobs and a set of computers to achieve enhanced performance. A detailed 
description of SmartNet is available elsewhere [FREU98, KIDD96]. 

Although SmartNet was designed to be used in actual production, the project did 
make significant research contributions. MSHN, on the other hand, is intended to be a 


research system that expands upon SmartNet’s research by considering the overhead that 
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job sharing resources have on mapping and scheduling algorithms, support of adaptive 
applications, and the delivery of good QoS to different sets of simultaneous users 
[HENS99]. While SmartNet focused on wall-clock time to determine estimated time to 
compute, this thesis discusses the monitoring of a variety of resources that impact QoS. 
The work most closely related to this thesis of which we are aware is that of MAJ 
Bob Armstrong, a graduate of the Naval Postgraduate School, whose thesis investigated 
the effects that different run-time distributions have on the performance of SmartNet. 
Armstrong used NAS benchmarks to determine the types of run-time distributions that 
might represent selected jobs on selected machines. Once these run-time distributions 
were determined, their parameters could be reproduced by a SmartNet simulator that he 
built. Experiments with benchmarks were run with the executable program, input and 
output data residing on the executing machine, as well as with the executable and data 
being obtained from a file server. When the executable and data were obtained from a 
file server, experiments were run with both a heavily and lightly loaded server and 
network. The executable programs used in these experiments, which involved sorting 
algorithms, were run both using a single processor of a multiprocessor computer and 
using multiple processors on the same machine. [ARMS97] The focus of these 
experiments was on total wall-clock time. Finer grained data regarding cpu time, number 
of bytes transferred via interprocess communication, number of page faults, and amount 
of data held in local cache were not collected. As a follow-on to SmartNet and 
Armstrong’s work, MSHN investigators have recognized the need to collect finer grained 
data, through the use of wrappers, in order to more closely estimate a variety of resource 


usage distributions. 


i. SUMMARY 


Section A of this chapter discussed the advantages of, and challenges inherent in, 
distributed systems. Section B described three middleware standards that are currently 
being investigated for their potential integration into a resource management system. 
Section C provided an overview of the MSHN project, and Section D discussed the 


related work of Bob Armstrong and the SmartNet project. 


psi] 





Iii. EADSIM 


This thesis presents the methodology and results of wrapping EADSIM as a 
representative, contemporary military application. It is thought that more finely grained 
data collected from a real application, in this case from wrapping EADSIM, will be far 
more valuable than wall-clock data collected from benchmark programs. Resource usage 
data, collected from EADSIM using the MSHN wrapper, will be used to determine 
distributions for a variety of computing resources. This data can then be applied to 
follow-on research using the MSHN emulator to investigate the performance of the 
various scheduling algorithms. 

Extended Air Defense Simulation (EADSIM), formerly known as the Command, 
Control, Communications, and Intelligence Simulation (C3ISIM) and also as the Theater 
Missile Defense (TMD)/C3ISIM, is a technology outgrowth of a computer-based 
modeling program developed in the late 1980’s. EADSIM is an analytic, Monte Carlo 
modeling tool for joint and combined force air and missile warfare. It is useful for 
scenarios ranging from few-on-few engagements to theater applications. It embodies a 
workstation-hosted, system-level simulation used to assess the effectiveness of air and 
surface launched missile defense systems against a spectrum of air and surface-launched 
threats. A robust’ C4I infrastructure for both aggressor and defender forces can be 
simulated, allowing planners to evaluate their own forces and enemy forces in either role. 
EADSIM is managed by the Testbed Product Office, Space and Missile Defense Battle 
Lab, US Army Space and Missile Defense Command as the executive agent for the 
Ballistic Missile Defense Organization (BMDO). EADSIM, selected as a primary 
simulation in the BMDO Tactical Missile Defense Cost and Operational Effectiveness 
study, is the primary mission level model for the Air Force, and was chosen as the 
primary single integrated operational plan (SIOP) analysis model for United States 
Strategic Command (USSTRATCOM). EADSIM is used by over 370 agencies, 


including all US services, DoD and other US government agencies, as well as by the 


7 Although in computer science “robust” 1s a software engineering term meaning that a program has been 
carefully engineered and tested to ensure that it is bug-free, this thesis uses the term in a more military 
sense. In this thesis, robust is used to mean a complex range of capabilities. 
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governments of the United Kingdom, the Netherlands, Israel, Australia, Republic of 


Korea, Japan, Spain, and NATO alliance countries. [SMDC97] 


A. OPERATIONAL ARCHITECTURE 


EADSIM displays scenario generation, preview, and post-execution information 
to the user through the use of a graphical user interface (GUI). Graphics provide full 
color terrain data with overlayed scenario icons, three dimensional displays of scenario 
playback from any location in the battlespace, previews of scenarios with flight paths, 
cross sections through terrain, attacker-to-target pairings, sensor intervisibility displays, 
overlayed text windows, and overlayed maps on terrain. The displays are easy to access 
and convey important, user-selected information. 

The GUI provides a series of pull-down menus as well as many point and click 
windows to view specification and input data. Help screens are available as needed. 
These screens give a short description of the specific input area, including appropriate 
units and examples. All data can be added directly by highlighting a field using the 
mouse. The GUI offers graphics for generating, modifying, playing back, and analyzing 
scenarios. These graphics are capable of presenting simulation results in two 
dimensional and three dimensional displays simultaneously, from anywhere in the 
battlespace. They can also be used to produce tailored textual results of engagements, 
launches, kills, communications, and detections. Additionally, EADSIM offers a bounds 
checking feature that includes contextual and consistency checks. 

EADSIM has the capability to interact (confederate) with other simulations using 
the Distributed Interactive Simulation (DIS) protocol standards, and the Aggregate Level 
Simulation Protocol (ALSP). It can be confederated with campaign level models such as 
the Corps Battle Simulation and Vector-In-Commander (VIC), with high fidelity models, 


and with virtual simulators. 


B. SYSTEM ARCHITECTURE 


EADSIM consists of three basic elements: simulation set-up; runtime models; 


and post-simulation analysis (Figure 3. 1). 
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Figure 3. 1 The Three Elements of EADSIM. After Ref. [METH98] 


A GUI executable provides the primary user interface for simulation set-up, post- 
processing and analysis tools. The four run-time processes are: Command, Control, 
Communications and Intelligence (C3I); Flight Processing (FP); Detection (Detect); and 
Propagation (Prop)(Figure 3. 2). The C3I process is event driven and serves as the 
simulation “driver,” performing C2 decision processing, track processing, message 
processing, and engagement and weapons modeling for all platforms in the scenario. The 
other three run-time processes are time-stepped, receiving input events generated by the 
C3lI process. Flight Processing maintains and updates the movement of aircraft, ballistic 
missiles, satellites, and surface platforms, providing “ground truth” to the other three 


processes. 
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Figure 3.2 The Four Run-Time Processes of EADSIM. After Ref. [METH98] 


The Detection process models each sensor specified within the scenario provided and 
determines when detection of each participant by each sensor occurs. Propagation 
models communications connectivity and propagation. Propagation can be used to 
determine when message transfer occurs and the effects of information flow on the C4l 
decision cycle. Since the Propagation process is potentially very compute and resource 
intensive, scenarios may be run by the other three processes exclusively, ignoring the 
Propagation process. To minimize run time and computing resources, many EADSIM 
users choose not to include the Propagation process in their simulations. (Of note, the 
scenario we used in the experiments described in Chapter IV was provided in the 
EADSIM installation package and does not use the Propagation run-time process.) 
Operations and platform types modeled include: 

e Air Defense (SAM, artillery, Command/Control); 

e Offensive Air Operations (CAS, SEAD, C2, TBM, Refueling, etc.); 

e Attack Operations (Surveillance, C2, Intel gathering, movement, etc.):; 

e Multistage ballistic missiles, Air Breathers (aircraft, missiles, helicopters); 

e Sensors (Radar, IR, SIGINT, IMINT, HUMINT, etc.); 

e Jammers; 

e Satellites; 


e Early Warning; 
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e Generic Noncombatants; 

e Communications (Networks, devices, messages); 

e Electronic Warfare (full scope of capabilities); 

e Terrain (masking, communications propagation); 

e Weaponry (air to air, surface to air, air to surface, surface to surface); and 


e Geographic Areas of Interest (MEZ, FEZ, AOR, etc). 


C. TECHNICAL ARCHITECTURE 


The execution of a scenario 1s performed by the four run-time models running in a 
multi-process configuration. Each process models some aspect of the scenario and 
exchanges data via interprocess communication during the scenario execution. 
Interprocess communication between the run-time processes is accomplished using 


sockets (Figure 3. 3). 
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Figure 3. 3 EADSIM Interprocess Communication via Sockets. After Ref. 
[METH98] 


Sockets are analogous to files and their use 1s analogous to file input and output 
(I/O) operations. When sockets are opened (using the system function call socket( )) 
they are named and associated with other sockets, thereby creating socket descriptors 
similar to file descriptors. Just as read( ) and write( ) operations are commonly 


used to modify and update files, read( ) and write( ) operations (perhaps more 
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aptly described as “send and receive” operations) allow data to be transferred between 
sockets. Following a close( ) operation, both file and socket descriptors are released 
back to the system. While files may be saved to memory and accessed later by a unique 
name, this is not the case with sockets. Socket names do not persist in memory following 
a close( ) operation. After performing a close(_ ) therefore, an application must 
again open a socket(s) if it subsequently needs to transfer data. Simply put, the purpose of 
sockets is to allow two or more processes on a single computer, or on computers 
connected via a network, to communicate with each other. [QUIN96] 

By using sockets for interprocess communication, EADSIM’s run-time processes 
can run on multiple computers. EADSIM’s sockets are blocking, that is they are 
established so a process will wait at the “read” operation until all data are received. This 
blocking mechanism allows for the proper sequencing of the processes. The timing and 
sequencing of the run-time processes are crucial to the ability of these processes to 
execute correctly. EADSIM primarily supports Monte Carlo simulation, but can also 
support deterministic analysis. 

Although EADSIM is written in C, its coding is object-oriented-like. The object 
oriented nature of the code has made revisions and improvements more straightforward. 
Future versions of EADSIM are planned to integrate the functionality of the four run- 
time processes into one executable, simplifying the code and improving the application’s 
performance on a single machine. 

EADSIM executes on either Silicon Graphics Workstations with Silicon Graphics 
operating systems 5.3 through 6.3 or on Sun Workstations (SPARC Station 10, 20, or 
ultra series) with Solaris 2.3, 2.4, 2.5, or 2.6 operating systems. It requires 1GB Disk 
Space, 192 MB RAM, a CD ROM, a 4mm tape drive, and dbx (a debug utility used to 


trace the execution of a process). 


1. Scenario Generation 


Scenario generation in EADSIM is a complex iterative process in which laydown 


files are constructed from groupings of platforms (Figure 3. 4). Platforms are players in a 
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scenario that reflect the attributes of prototype players. These prototype players, or 


system elements, 
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Figure 3. 4 Hierarchy of Data Builds an EADSIM Scenario. After Ref. [TELE98] 


represent aggregates of elements (low-level component definitions). Element types 
include sensors (coverage and capabilities), airframes (flight dynamics), jammers (power, 
frequency, bandwidth, coverage), weapons (performance and effectiveness), fly-out 
tables (flight characteristics of interceptors), radar cross sections, infrared signatures, 
probability of kill (geometry and target dependent kill probability), power propagation 
tables (for directed energy weapon propagation), flight formations and relationships 
(command chain), icons for display, communication devices (power, frequency, 
bandwidth, coverage), protocols (message size, priority, and maximum life), rulesets 
(platform behavior), and classes (groupings of types of targets for identification). 
Complete details of the construction of scenarios are contained in the EADSIM v7.00 


Methodology Manual. [METH98] 
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D. SUMMARY 


EADSIM supports the "four pillars" of Theater Missile Defense (TMD) by 
modeling Active Defense, Passive Defense, Attack Operations, and Battle 
Management/C3I (including engagement logic, Command and Control structure, 
Communications networks, and protocols). By modeling Theater Ballistic Missile 
Defense and Air Defense in a dynamic environment, EADSIM offers analysts and 
training personnel enhanced insight into TMD architecture, battlke management, system 
employment for maximum effectiveness, force structure analysis, and mission planning. 
[SMDC97} 

The ability of EADSIM to serve as a timely decision support tool for the 
Warfighter has been demonstrated (Chapter II, Section B, Subsection 2 discusses 
EADSIM’s inclusion in the COMPASS architecture) in Joint and Service sponsored 
exercises, Joint Warrior Interoperability Demonstrations, and Fleet Battle Experiments. 
Use of the model to support ATO preparation and planning during Operation Desert 
Shield/Desert Storm and its current use by more than 370 agencies worldwide attest to its 
potential as a decision support tool in a distributed, C4I planning environment. 
EADSIM is representative of modeling and simulation tools that can be made more 
accessible and timely through an ability to adapt to dynamic environments. EADSIM’s 
ability to run deterministic as well as stochastic simulations made it a logical choice for 
measuring and comparing the computing resources needed to execute these two 
“versions” of the program. Additionally, EADSIM offers the ability to create elements 
with varying degrees of fidelity integrated with command and control chains consisting 


of varying degrees of complexity. 
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IV. EXPERIENCES WRAPPING A C4I MODEL 


This chapter describes experiences with wrapping a C4I-modeling application, 
EADSIM. The reason for wrapping this application 1s two-fold: 1) to acquire resource 
usage information from a non-trivial, military, command and control application, and 2) 
as a proof of concept of the transparency of MSHN’s wrappers. MSHN has developed a 
method of intercepting an application’s requests for hardware services via the operating 
system in order to measure the computing resources required to execute the application 
[SCHN98]. This method is called “wrapping” the application and enables MSHN to 
measure computing resource usage without incurring significant overhead and without 
requiring access to the application’s source code. MSHN’s wrappers can be linked with 
application object code without requiring modifications to source, access to source, or 
modifications to the operating system. Therefore, it is unlike DeSiDeRaTa [UTAR96], 
Graze [MOOR95], and Pablo [REED96] which all require source modification. Unlike 
these tools, MSHN’s wrappers obtain both the application’s resource requirements and 
the current status of the resources by intercepting operating system calls. 

Section A of this chapter provides a high level explanation of how MSHN wraps 
operating system calls. Section B discusses the methodology of wrapping the C4I-model, 


EADSIM, including steps taken to correct minor problems when encountered. 


A. INTERCEPTING SYSTEM CALLS WITH “WRAPPERS” 


Operating systems serve two primary purposes: to fairly allocate hardware 
resources that must be shared among applications and, to “beautify” the hardware by 
providing the user a higher level interface with which to access the computer’s hardware 
resources. The interface between the user and the hardware takes the form of system 
calls. When an application needs a hardware resource (i.e., disk space or the network) a 
call is made to a user level library. This user level library, which is linked with the 
application, requests the hardware on the application’s behalf by invoking a system call 
from the operating system. All UNIX applications, for instance link, with the C library, 


which then makes the system calls on their behalf. 


a 


Based on research done for the SmartNet project, MSHN investigators determined 
that to more intelligently manage resources in a distributed system, fine grained resource 
usage data would have to be automatically collected and analyzed by the resource 
management system (RMS). Specifically, MSHN’s Client Library is responsible for 
collecting and distributing this data within MSHN. A method was needed that could 
gather data without incurring excessive overhead. That is, the method must not tax the 
very resources it was monitoring. 

In his thesis [SCHN98], Matthew Schnaidt described in detail the method he used 
to intercept system calls, which was derived from a mechanism used by CONDOR 
[LIVN95], thereby providing a means to measure resource usage through classes within 
the Client Library [SCHN98]. This method involves “wrapping” an application within a 
composite library (that includes a modified C library), thereby allowing requests for 
hardware to be intercepted before they reach the operating system. Similarly, values that 
are returned by the operating system call can be analyzed by this modified library. The 
library of functions that perform this service, by linking with an application to be 
monitored, are referred to as an application “wrapper.” An initial determination of 
which system calls to intercept in order to collect fine enough grained data for the 
resource management system to perform optimally was also addressed in Schnaidt’s 
thesis. Experience with SmartNet demonstrated that the estimated time to compute 
(ETC), based on measuring wall-clock time, was insufficient to accurately predict an 
application’s resource requirements, and hence its total run-time when executed in a 
different environment. To better understand the effect various resources have on the 
execution of a process, MSHN investigators identified key resources to monitor and the 
metrics associated with them (Table 1). 

It was initially determined that the major resources to monitor include the total 
time an application controls the cpu (cpu time), the total main memory and cache 
memory used (measured in pages), local and remote disk access (measured in number of 
bytes), terminal I/O (measured in bytes and time spent blocked awaiting user input), 
interprocess communication and other network traffic (both measured in bytes 
transferred). In order to gather resource usage data on these resources, several system 


calls need to be intercepted by the Client Library wrapper. The system calls that need to 
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be intercepted by MSHN’s wrapper functions include socket( ), connect( ), 
open ( ), close ( ),pipe( ), read ( ),write ( ), andexit ( ). 
Additionally, in order to wrap an application’s start-up a modified MAIN( ) function 
was written. A detailed description of how the wrapper was written and implemented is 


beyond the scope of this thesis, but 1s provided elsewhere [SCHN98]. 
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Table 1: Resources to Monitor. From Ref. [SCHN98] 





B. WRAPPING EADSIM 


The process of wrapping an application begins with identifying which system 


calls are to be wrapped. A copy of the C library functions corresponding to these system 
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calls then needs to be modified (in the case of MSHN wrappers, the names of system call 
functions contained in the C library are simply converted to upper case to distinguish 
them from their wrapper function counterparts). Wrapper functions, by the same name as 
the original system call function, must then be written for each system call to be wrapped. 
These wrapper functions will invoke the original system call by using its modified (upper 
case) name, and will perform additional resource monitoring and measurement 
functionality as required. The wrapper functions are then linked with the modified C 
library resulting in a composite client library. The C run-time object file containing the 
function call main( )is modified in a similar manner (with the name main( ) being 
changed to MAIN()). The MSHN Client Library and the modified C run-time object 
file are then linked with the application’s object file(s). The wrapping of the application 
is then complete. 

In order to wrap EADSIM as a proof of concept of MSHN’s Client Library 
wrapper functions, it was necessary to obtain the application’s object code®. Recall from 
Chapter II that MSHN’s wrappers were designed to be transparent to the user, that is to 
be executed without requiring any modification to the user application’s source code. 
With the US Army’s Strategic Missile Defense Command providing release authority, 
Teledyne Brown Engineering (TBE) delivered the object and archive files ( .o and .a 
files) for EADSIM v7.00, as well as a makefile. Using the README files provided with 
the MSHN Client Library source code, and the Tutorial on Wrapping System Calls 
originally written by Matthew Schnaidt [SCHN98] and later modified for use with the 
Solaris 2.5 operating system by MSHN staff member Shirley Kidd [SKID99], we 
modified the README-FIRST file (APPENDIX A) and proceeded to wrap EADSIM by 
following the steps noted in paragraph 3 therein. 

We first created a modified C library, libMSHNC.a, and copied this into the 
syscall_lib directory that contained the files of the MSHN Client Library. Ensuring that 
the MSHN_types.h file defined variables for the Solaris 2.5 operating system (SUN5), we 
ran the libraryMakefile to compile the client library, which produced the 


8 Theoretically, it should be possible to wrap executables instead using the Executable Editing Library 
(EEL) [LARU95] tool developed by the University of Wisconsin’s Paradigm project. However, doing so Is 
the topic of another MSHN thesis. 
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MSHN_syscall_lib.o file. The MSHN_syscall_lib.o file was then copied into the 
EADSIM directory for later linking with the EADSIM object files. At this point we were 
unaware of modifications that would be required to the files MSHN_syscall_lib.cc, 
MSHN_Monitor_RRD_CLASS.cc, and libraryMakefile in order for the client library to 
successfully wrap EADSIM. To catch calls tomain( ), we next modified the C run- 
time object file, crtl.o, and renamed it Mod_crtl.o. This file was also copied to the 
EADSIM directory. 

After modifying the original EADSIM makefile provided by TBE to reflect local 
directory paths, we inserted code to link EADSIM’s object files with 
MSHN_syscall_lib.o and included the “-###’ command so that the C language 
compiler, “cc”, would show each component as it was invoked without actually 
executing (operating in the verbose mode without producing executables for each 
process). The output from this modified EADSIM makefile was used to produce 
separate (c shell) csh scripts to compile and link each run-time process. These individual 
EADSIM-process csh scripts were then modified by substituting the C run-time object 
file (crtl.o) with the modfied C run-time object file (Mod_crtl.o). It was later realized 
that to successfully link the EADSIM object files (recall that EADSIM is wnitten in C) 
with the MSHN_syscall_lib.o file (written in C++), the makefile used to produce 
MSHN’s syscall_lib.o object file would have to explicitly add a reference to the directory 
containing the stdc++ library to the list of directories to be searched by the linker. 

EADSIM v7.00 was installed, from the production CD, on a SUN SPARC 20, a 
SUN Ultra One, and an SGI Indy workstation. The modified EADSIM-process csh 
scripts produced executables that appeared to be successfully wrapped. The four run- 
time processes provided with the installation CD (C3I, FP, Detect, and Prop) were then 
replaced with the wrapped run-time executables. These executables were used to run the 
Demo300 scenario provided in the EADSIM v7.00 installation package (use of the 
Demo300 scenario is described in more detail in Chapter V). | While the simulation 
executed successfully, the wrapper did not initially produce output files as expected. 
Troubleshooting and debugging were needed. 

After extensive debugging of the MSHN client library source code and EADSIM 


csh scripts, it was learned that the following modifications were needed: (1) we needed to 
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explicitly add the “libstdc++.a,” the “libstdg++.a,” the “libm.a,” and the “libc.a” to the 
library Makefile (APPENDIX B) used to produce MSHN’s syscall_lib.o object file so 
that C++ language symbols used in the Client Library would be defined in the EADSIM 
executables at compile time; (11) DEBUG _— statements’ defined in 
MSHN_Monitor_RRD_Class.cc, that had been added to the write( ), _write( ), and 
__write( ) functions after the program had undergone testing, were causing 
segmentation errors and needed to be removed; and (111) the correct invocation of the 
Unix -ps (process status) command had to be incorporated into the getPsData ( ) 
function of the MSHN_Monitor_RRD_Class.cc code (APPENDIX D). Once these 
changes were made, csh run-scripts were developed to properly reset the random number 
generator seed for each run of the Demo300 scenario (APPENDIX E), and another run- 
script was written to transfer the data from each run (both simulation outcomes and 
wrapper Output) to a separate directory for later analysis (APPENDIX F and APPENDIX 
G). 

During the debugging process, it was immediately evident that functions deriving 
from the C++ iostream library were not being recognized (nor were any of the 
functions from the C++ strings class). The ostream class object cout, for 
instance, did not output debug statements to the screen. The C language function, 
print£ (), however, did perform as expected. It was determined that the original files 
in the MSHN Chient Library, written in C and C++, had always been compiled (and 
linked) with C++ language test programs, using a C++ compiler which automatically 
linked the MSHN_syscall_lib.o object file with the libstdc++.a library. However, when 
attempting to link the MSHN_syscall_lib.o object file with the EADSIM object files 
(compiled using a C compiler), to produce an executable, the C++ lbraries were not 
being searched, and symbols being referenced in the MSHN_syscall_lib.o file remained 
undefined. This was verified by running the print name list of an object file (nm —-f) 
utility on the EADSIM wrapped executables. The nm utility displays the symbol table of 
each ELF object file that is specified by the input file argument and will report if no 
symbolic information is available for a valid input [BERK91]. By running the nm —-f 
command on the EADSIM executables it was discovered that several C++ symbols were 


undefined. Using the g++ compiler with the —v option (verbose mode) we compiled a toy 


42 


program and noted that the libstdc++.a library, the libg++.a, the libm.a, and the libc.a 
libraries were referenced in the output. We then found the local path to these libraries 
and explicitly referenced them, using the -L command, in libraryMakefile2 (APPENDIX 
B). By recompiling the MSHN Client Library with this makefile, we produced a 
MSHN_ syscall_lib.o file that, when linked with the EADSIM object files, produced an 
executable with statically defined C++ symbols. The wrapper then successfully printed 
debugging statements to the screen and produced output files as expected. 

Apparently, after the MSHN Client Library had been successfully tested by 
Matthew Schnaidt{SCHN98], DEBUG statements were added to many of the functions in 
the MSHN_Monitor_RRD_Class.cc file to assist anyone planning to implement the 
Client Library in the future. It was found, however, that when DEBUG statements were 
defined within the MSHN_Monitor_RRD_Class.cc, running the wrapped executables 
caused a segmentation fault. This segmentation error was due to recursive calls to 
write() when invoking cout within the write(), _write() , and __write() 
functions in the MSHN_syscall_lib.cc file. The following lines were removed from 


these functions: 


#ifdef MSHN_DEBUG 
cout << “CAUGHT A __wnite()” << endl; 

#endif 
This solved the segmentation error problem and provided a lesson on attempting to 
invoke a write () system call from within a write ( ) function. 

Once the changes were made to the Client Library makefile and to the 
MSHN_syscall_lib.cc file, the wrapped executables produced output files containing 
resource usage data. However, this data clearly was incorrect (identical output values 
were being produced by two of the three functions). There was something in the 
getPsData () function, in the MSHN_Monitor_RRD_Class.cc file, that was not 
returning correct values. The problem was found to be in the invocation of the report 
process status(ps) command. The ps command prints information about active 
processes [BERK91]. The original MSHN_Monitor_RRD_Class.cc code invoked the ps 


function using the group list (-g) option, which prints information about all processes in 
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a group list. The ps utility is used inthe getPsData () function to determine virtual 
memory in kilobytes used by each wrapped process ( -vsz), the number of physical 
pages in memory (-osz) for each wrapped process, and the number of pages of memory 
present in the resident set ( -rss) of each wrapped process. It was determined however, 
that the —g option did not return data for each of the three executing processes. The —g 
option was replaced with the process list (-p) option, which returns data only on 
specified process ID numbers (pid) listed in the process list. With this modification made 
to the get PsData() function in MSHN_Monitor_RRD_Class.cc, the MSHN Client 
Library was re-compiled and the MSHN_syscall_lib.o file was relinked with the 
EADSIM object files. This produced the desired system usage data for each wrapped 
process. 

One additional change to the code was made before recompiling the MSHN 
Client Library. The MSHN_syscall_lib.cc file originally written by Matthew Schnaidt 
[SCHN98] was intended to monitor processes communicating via network interprocess 
communication (IPC) as well as local interprocess communication. Because EADSIM’s 
run-time processes would be run on one workstation, it would not be necessary to 
measure network latency and throughput by monitoring network IPC. To avoid adding 
overhead needlessly (by checking each connect( ) and accept( ) system call to see 
whether it was a local or network IPC), since all connect( ) and accept( ) system 
calls made by the EADSIM run-time executables would be made via local IPC, code was 
modified within the acceptWrapper( ) and connectWrapper( ) functions to 
ensure that all interprocess communication would be interpreted as local IPC 
(APPENDIX C). An alternate approach to reducing overhead is discussed in Matthew 
Schnaidt’s thesis [SCHN98], in which he suggests that follow-on research might 
incorporate Synthetix tools, to dynamically optimize the MSHN Client Libraries at run- 


time. Synthetix tools are described in detail elsewhere [PUCA96A] [PUCA96B]. 


OF SUMMARY 


Section A of this chapter discussed “wrapping,” a method of intercepting system 


calls in order to monitor resource usage. Section B described the steps taken to wrap 


EADSIM’s four run-time processes, C3I, FP, Detect, and Prop. Problems identified 


through debugging were discussed, and solutions to these problems were explained. 
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V. PROVIDING GOOD QUALITY OF SERVICE WITH LIMITED 
RESOURCES IN A C4I MODELING APPLICATION 


Section A of this chapter discusses the reasons EADSIM is a logical candidate to 
consider when determining whether good Quality of Service (QoS) can be provided when 
resources are limited. Section B describes the air and missile warfare scenario used in 
our experiments. Section C discusses the measures of effectiveness (MOE) chosen to 
compare the outcomes from the deterministic simulation with the outcomes from the 


Monte Carlo (stochastic) simulations. 


A. WHY EADSIM? 


Recall from Chapter II that MSHN’s goal is to provide a resource management 
system (RMS) to enable applications that contend for shared resources to obtain good 
QoS while accounting for dynamically changing priorities and environments. Operating 
in a heterogeneous computing environment in which many processes are competing for 
shared resources, an RMS such as MSHN, must be capable of providing each subscriber 
process with its required Quality of Service (which might include security considerations, 
deadlines, user priorities, and preferences). The MSHN architecture is designed to be 
integrated with, and to incorporate, a variety of distributed system tools (1.e., CORBA, 
ENSEMBLE, COMPASS, etc) in order to take advantage of all available resources. 
Applying this technology to C4I modeling and simulation applications would enable on- 
scene Commanders to simulate complex elements of the decision process in order to 
optimize the use of forces and materiel. In order for the RMS to offer this flexibility in 
response to user defined Quality of Service parameters, modeling and decision support 
applications must be adaptive. That is, they must exist in different versions capable of 
producing like results (though possibly offering varying degrees of Quality of Service). 

EADSIM is an air and missile warfare modeling and simulation application that 
offers great flexibility in (1) the areas modeled, (11) the capabilities of the platforms 
simulated, and (iii) the method of simulation (deterministic or stochastic). When building 


a scenario, each platform (i.e., aircraft, missiles, sensors) is modeled individually as is the 
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interaction among platforms. A robust range of characteristics is offered for each 
platform, allowing the scenario developer(s) the opportunity to select the granularity they 
need to model in order to produce a result with sufficient fidelity to suit their purpose. 
Users may chose to simulate the Command and Control (C2) decision process and the 
communications among platforms on a message-by-message basis, simulating message 
traffic flow through command networks and the impact this flow has on the decision 
process. Intelligence gathering is also modeled, as is the flow of intelligence 
disseminated to the commanders making operational decisions. All of these capabilites 
can be applied to both offensive and defensive operations, allowing analysts and planners 
to evaluate strike and counterstrike scenarios textually, in 2-D playback and/or 3-D 
playback. It was EADSIM’s flexibility, and potential as an adaptive C4I application, that 


made it a logical choice as a proof of concept for MSHN. 


B. THE DEMO300 SCENARIO 


Due to the complexity inherent in designing and building a realistic scenario that 
would exercise many of EADSIM’s capabilities, it was decided to use one of the 
demonstration scenarios provided with the EADSIM v7.00 installation package for our 
experiments. Discussions with Teledyne Brown Engineering representatives led to the 
selection of the Demo300 scenario, due to its broad range of platforms modeled, its 
plausible force layout and mission objectives, and the short duration (wall clock time) of 
each simulation. The only drawback noted in choosing Demo300, was that the scenario 
did not model communications propagation (did not use the “Prop” run-time process). It 
was explained to us by the TBE engineers that EADSIM users frequently elect not to 
execute the propagation process because of its considerable computing resource 
requirements. Communications in Demo300 were then considered “perfect” with 
network latency, contention, and congestion not being simulated. 

Demo300 simulates a large offensive air strike with a leveraging strike of tactical 
missiles. Air launched cruise missiles and lofted trajectory TBMs are targeted against 
Command and Control (C2) nodes--for both SAMs and Defensive Counter Air (DCA), 


and the Intelligence Center, while depressed trajectory TBMs are targeted against the Air 
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Base itself (Figure 5.1). With SAM sites at least partially suppressed, offensive air-to- 
air fighters can engage defensive counter air fighters and divert them from the offensive 
ground attack aircraft that are to follow (Figure 5. 2). This leveraging strike 1s intended 
to soften the defenders’ ability to counter the bulk of the attack conducted by ground 
attack aircraft. A stand-off jammer aircraft attempts to blind the defenders’ sensors and 


disrupt communications links. 
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Figure 5. 1 Offensive Missile Targets. After [EADS98}] 


To counter this attack, a number of actions will be taken, utilizing a panoply of 
defensive and counter-offensive assets. Early warning and ground surveillance aircraft, 
battlefield C2 sensors, a lJow-observable, reconnaissance fighter aircraft, and an 
intelligence collection satellite will provide alerting information and target cueing for 
counter-offensive operations (Figure 5. 3). SAM and DCA C2 nodes will prioritize and 
check targets for engagement based on the available consolidated air picture, areas of 
responsibility, capabilities of assigned assets, and perceived threat to the assets being 
defended. SAM and DCA C2 nodes will perform deconfliction in the case of dual or 


multiple engagements of a single ingressing missile or aircraft (Figure 5.4). While the 
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air defense operations are ongoing, ground targets will be selected for counter-offensive 
operations. 

Aided by intelligence, reconnaissance and surveillance assets, the defending force 
will target the attacking force’s ground components that are supporting the attack. These 
ground “targets would include ballistic missile launchers, sites occupied by these 
launchers, and the command chain controlling the launchers (Figure 5. 5). Hostile 
Division headquarters would also be targeted. Defensive force assets used in this counter 


offensive consist primarily of mobile rocket systems (MRS). 
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Figure 5. 2 Offensive Strike Aircraft and Tactical Missiles. After [EADS98] 
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Figure 5. 3 Defensive C2 Sensors, Reconnaissance, Surveillance and Early Warning 
Aircraft. After [EADS98] 
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Figure 5.4 Deconfliction is Coordinated Among the Defensive SAM and DCA 
Commanders. After [EADS98] 
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Figure 5.5 Counter Offensive Targets. After [EADS98} 


C. CHOOSING A MEASURE OF EFFECTIVENESS 


Section C of Chapter II and Section A of this chapter, have discussed the fact that 
fundamental to MSHN’s design as a resource management system is the capability of 
providing each subscriber process with its desired Quality of Service (QoS). QoS 
considerations commonly sited include security, deadlines, user priorities, and 
preferences. To military commanders and planners, these QoS considerations represent 
very real constraints that vary with the situation and may mean the difference between 
success and failure, life and death. 

MSHN investigators are currently developing a framework for a performance 
measure that can be used for scheduling resources in a distributed heterogeneous 
environment. Such a performance measure would need to assess the ability of the RMS 
to simultaneously satisfy, to varying degrees, QoS requirements. QoS attributes 
considered critical to analyzing and evaluating a system’s performance are priorities, 


versions (which represent a user’s preferences), deadlines, and security. These attributes 
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are defined, and their role in developing an optimization criterion called the Flexible 
Integrated System Capability (FISC) ratio is discussed in detail elsewhere [JONG99}. 

In a military environment, timeliness, security considerations, and priorities 
dictated by the operational situation greatly influence the decision-making process. 
While a commander or planner may have little control over these variables, it is rarely the 
case that decisions are faits accomplis, offering no opportunity to apply heuristics to 
alternative courses of action. Just as EADSIM is one tool intended to be used in the 
evaluation of alternative courses of action, it 1s an example of an application that could 
potentially be used in several variations, providing the user the opportunity to state a 
“preference” for one version over another. As stated in Section A of Chapter I], the key 
to implementing an RMS that can support QoS requirements in a distributed, 
heterogeneous computing environment, is the availability of adaptive, or adaptation- 
aware, applications. It is not the purpose of this thesis to examine the potential of 
EADSIM as an adaptive application. Several characteristics of EADSIM allow the 
application to provide the user with a variety of choices, in order to support the 
constraints of the decision making environment. 

Among the characteristics of EADSIM that may be manipulated to support 
varying time and resource constraints are (1) the application’s ability to display the 
scenario playback in either 3-D or 2-D graphics or to simply provide simulation results 
through a variety of tailorable, textual reports; (ii) the ability to input varying degrees of 
fidelity for each platform’s characteristics, thereby “scaling” the model’s scope; (111) the 
ability to model communications propagation with varying degrees of fidelity, or to 
simply model “perfect communications,” avoiding the considerable computing resources 
needed to execute the propagation run-time process; and finally, (iv) the ability to execute 
the application as either a stochastic or a deterministic simulation. 

A primary objective of this thesis was to wrap a complex C4I modeling 
application in order to determine the resources required to execute the application on a 
stand-alone workstation in two different configurations, or “versions,” and to compare 
the results obtained from each model configuration against the computing resources 
required to run them. We therefore sought a reasonable measure of effectiveness with 


which to compare different “versions” of EADISM. 
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In order to make our proof of concept as manageable as possible we chose to 
wrap the four run-time processes (despite the fact our experiment did require the use of 
one of these). As discussed in Section B of this chapter, we chose to use a demonstration 
scenario provided with the EADSIM V/7.00 installation CD. The use of the Demo300 
scenario, which modeled perfect communications propagation (and therefore did not use 
the prop run-time executable), eliminated the possibility of comparing a version of the 
simulation running the propagation executable with a version that did not. However, 
even this “example” was quite complex, corresponding to more than 600 pages of 
scenario platform laydowns. 

One aspect of EADSIM however, made it particularly attractive as an example of 
a potentially adaptive application. That characteristic was the application’s ability to 
support both stochastic and deterministic simulation. The Monte Carlo method of 
simulation is dependent upon the generation of random numbers and the use of 
probability distributions to simulate real world situations that contain an element of 
chance. Random number intervals, intended to represent possible outcomes for 
probabilistic variables in the model, are selected from a random number table or are 
generated by computer to simulate variable outcomes. The random nature of a stochastic 
simulation infers that each independent run produces only an estimate of a model’s true 
characteristics for a given set of input parameters. Therefore, a significant number of 
independent trials must be run in order to evaluate the probabilities of conditions being 
modeled. On the other hand, a deterministic simulation model, strictly defined, does not 
contain any probabilistic elements. The outcomes from a deterministic simulation model 
will not vary from run to run. [KELT91, REND97] 

EADSIM is primarily intended to be run stochastically. That is, it is a discrete- 
event simulation model that employs the Monte Carlo method to simulate the random 
nature of events ina scenario. A series of such Monte Carlo runs would then be required 
to adequately evaluate a given scenario. Depending on the complexity and scope of the 
scenario being simulated, this might require a significant amount of time and computing 
resources. However, EADSIM also offers the user the opportunity to run a scenario in a 


deterministic simulation, called the Planner Mode. 
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EADSIM’s Planner Mode is intended to allow the collection of data from a single 
scenario execution, rather than averaging data over a series of Monte Carlo simulation 
executions. With the Planner Mode, engaged platforms or ballistic missiles are not 
allowed to be destroyed; rather, the software accumulates probability of kill (Pk) and 
engagement geometry data against platforms as the scenario progresses. The C3] process 
runs without any platforms being killed. A penetrator aircraft is allowed to proceed from 
a designated beginning time, along its route, to a designated end time, accumulating 
probability of kill data as a result of weapon engagements, all the way to the designated 
target. [METH98] 

Attrition information is computed and logged for a variety of conditions. This 
data may be viewed using the Post Processing application, allowing planners to analyze 
single shot, cumulative, and route probability of kill information for all engagements. 
Single shot Pk is the Pk for a single shot against a target. Cumulative Pk is the aggregate 
Pk from all (single) shots taken against a target. Route Pk differs from cumulative Pk in 
that it includes the effects of weapon reliabilities as well as the attrition of the engaging 
platform. Consequently, the route Pk is usually less than the cumulative Pk, since single 
shot Pk will degrade as the attrition of the engaging platform increases. This data may be 
quite useful in analyzing a strike route, a strike “package,” or placement of defensive 
counter-air assets. 

Because no platform is destroyed, post-processing reports resulting from a run in 
Planner Mode are not readily comparable to post-processing reports generated from 
Monte Carlo simulations. This lack of similar reports was a challenge to the 
development of an acceptable measure of effectiveness that can be used to compare the 
results from the Planner Mode to the results from the Monte Carlo simulation. 

It was in examining one possible use of EADSIM that a measure of effectiveness 
suggested itself. If, it was reasoned, a commander or planner with severe time or 
resource constraints, was interested in determining the probability that at least one 
tactical missile (either an air launched cruise missile or a ballistic missile) would reach its 
intended target, he might want to run the Planner Mode for a quick look, worst case (or 
best case, depending on the user’s view point) estimate. To compute this probability 


from the Planner Mode (which, it must be stressed, is not the intended use of the 
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application), we deliberately ignored the model’s algorithms for determining the effects 
of engagement dependencies, and assumed all probabilities of kill against missiles were 
independent events. We then used the product of all final missile route Pk’s to determine 
the probability that every missile was destroyed just before it hit its target. We subtracted 
this value from 1 to determine the probability that at least one of the missiles was not 
destroyed: 


n 


I] (final route Pky = p(all missiles are destroyed) 


i=! 


p(at least one missile is not destroyed) = (1 - p(all missiles are destroyed)) 
m = probability of kill for the i" tactical missile (cruise missile or TBM) 
nm = number of missiles in the scenario 


This probability could then be compared with the observed outcomes from the Monte 
Carlo simulation. The MSHN wrappers would measure the difference in computing 
resources required to run a single Planner Mode simulation and a Monte Carlo simulation 


(consisting of a statistically significant number of runs). 


D. SUMMARY 


This chapter discussed the selection of EADSIM as a potential C4I adaptive 
application. The scenario chosen for experimentation was TBE’s Demo300 
demonstration scenario, which simulates an air strike on a defender’s air base with a 
leveraging strike from tactical missiles. The scenario was described in section B of this 
chapter. While EADSIM offers the user a variety of scenario generation, execution and 
post-processing options, it was important to find a measure of effectiveness with which to 
compare two versions of the application. The ability of EADSIM to run either stochastic, 
Monte Carlo method simulations or a deterministic simulation provided an opportunity to 
choose the probability that at least one missile from the aggressing force strikes its target 


as the measure of effectiveness. 
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VI. A DESCRIPTION OF THE EXPERIMENT 


This chapter describes the experiment designed to compare the resources required 
to execute EADSIM running both stochastic and deterministic simulations, to compare 
the results obtained from these two configurations of EADSIM, and to determine the 
overhead incurred by the MSHN wrapper. Section A of this chapter describes the 
computing environment in which the experiment was conducted. Section B discusses the 
experiment’s methodology, explaining how and why each of the three experimental runs 
was conducted. Section C presents the resource usage data and results obtained from the 
stochastic simulation, and Section D offers the resource usage data and results from the 
deterministic simulation. Section E provides the data used to determine the overhead 


incurred by the MSHN wrapper. 
A. HARDWARE AND OPERATING SYSTEMS 
I: EADSIM Hardware and Operating System Requirements 


EADSIM can either be run on either an SGI or SUN workstation with the 
minimum requirements noted in Figure 6. 1. It was found that EADSIM would execute 


with less RAM, 
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SGI SUN 


Machine Model: Indy Machine Model: UJltra-1, with Creator 3- 
D Graphics Card 

RAM: 256 MB RAM: 256 MB 

Hard Drive: 1 GB Hard Drive: 1GB 

Graphics: 24 bit planes, double Graphics: 24 bit planes, double 
buffering buffering 

OpenGL: Version 1.1] OpenGL: Version 1.1 

Operating System: 6.x Operating System: Solaris 2.5.1 (or higher) 


Window System: Common Desktop 
Environment (CDE)* or OpenWindows 


*recommended 


Figure 6. 1 EADSIM Minimum Hardware and Software Requirements as Stated in 
EADSIM User’s Guide. 


but that the C3ISIM GUI executable was not supportable on SUN workstations without 


the Creator 3-D graphics card. 


2: Equipment Used in Experiment 


EADSIM Version 7.00 was installed on a Silicon Graphics, Inc. (SGI) IRIX64 
server from the CD provided by the U.S. Army Space and Missile Defense Command. 
All executables and data files, including the demonstration scenarios, were then mounted 
on an SGI IRIX workstation, a SUN SPARC-20 workstation/server, and a SUN Ultra-1 
workstation (Table 2). Both SGI machines ran the SGI 6.2 operating system, while both 
SUN workstations ran the Solaris 2.5.1 operating system. These machines were 
connected by a 10 Megabit per second (Mb/s) Ethernet LAN. The SGI IRIX 64 file 
server runs SUN’s network file service (NFS). Neither the SUN Ultra-1 nor the SUN 
SPARC-20 workstation were equipped with a Creator 3-D graphics card, which required 
all pre-and post-processing of the model to be controlled from the command line, vice 
EADSIM’s GUI executable, C3ISIM. The SUN Ultra-1 was upgraded to include 256 
MB of RAM, while the SUN SPARC-20 workstation had only 98 MB of RAM. 


60 


It was possible to execute the EADSIM run-time processes on both the SUN 
workstations. Therefore, although EADSIM would execute the simulations on the SUN 
workstations and output log files and the “stathdr” file needed for post-processing the 
results of the run, this post-processing of reports could not easily be accomplished on the 
SUN workstations and, without the 3-D Creator graphics card, there was no means to 
display the scenario playback graphics on the SUN workstations. The EADSIM GUI 
was, however, accessible on the SGI workstation (despite the fact that this workstation 
was equipped with only 32 MB of RAM). By setting the environment path on the SUN 
workstations to access the scenario data files needed to run the simulation, and setting the 
resource path on the SGI workstation to access the log files and stathdr file produced by 


the executables, we were able to run the simulations on the SUN workstations and do 


post-processing and display the scenario playback graphics on the SGI workstation. 





SGI Server el Workstation SUN Workstation SUN Workstation 


Type Machine SGI Challenge L SGI Indy SUN Sparc-20 SUN Ultra-1 | 
Series : | : 


| Processor Speed | 200MHZ |  100MHZ ~ 50 MHZ 167 MHZ 
Processor Type MIPS R4400 MIPS R4000 TI,TMS390Z50 SUNW, 
—— es Lee Le | eemsre 
Number of 4 ] ] l 

Amount of 192 Mbytes 32 Mbytes 98 Mbytes 256 Mbytes 
ae a a ee 


Table 2: Configuration of SGI and SUN Workstations Used in Experiment. 













B. EXPERIMENT METHODOLOGY 


Because the population distributions of EADSIM’s resource usage were not 
known, it was necessary to estimate the mean and standard deviation of resource usage 


data collected by the MSHN wrapper. It was also necessary to estimate the mean and 
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standard deviation of the cumulative CPU times collected by EADSIM’s process 
statistics pstat facility? as well as the simulation trial outcomes themselves. The 
central limit theorem provides a justification for estimating a population mean based on 
the mean of a large sample of independent observations. The central limit theorem states 
that the’sum of a large number of independent observations from any distribution tends to 
have a normal distribution [JAIN91}. Using a large sample space (thirty trials), we 
estimated the population mean of each resource parameter measured, by obtaining the 
sample mean, standardizing it and using a standard normal table to obtain a confidence 
interval for the mean. This process also made it possible to estimate the mean of the 
simulations’ results, so that a comparison could be made between the probability of at 
least one missile reaching its target and the observed sample mean of at least one target 
being hit. 

The purpose of the experiment dictated the method in which it was conducted. 
The goals of the experiment included the following: (1) use the MSHN wrapper to 
measure requirements of a pre-determined set of resources (discussed in Chapter IV, 
Section A) for each of the three EADSIM executables running the stochastic simulation 
(employing Monte Carlo methods); (11) use the MSHN wrapper to measure the computing 
requirements of the same resources for each of the three EADSIM executables running 
the deterministic simulation (Planner Mode); (111) measure the overhead incurred by the 
MSHN wrapper by using the EADSIM pstat facility to measure the cumulative CPU time 
of both the wrapped and unwrapped executables running the deterministic simulation; 
and (iv) use EADSIM’s post-processing report generation to compare results from the 
stochastic simulation with results form the deterministic simulation. Each of these goals 
suggested a specific method of gathering data. 

An observation that was made early in the experiment design process, was that the 


number of page faults and the related size of the resident set in memory were directly 


9? EADSIM’s pstat functionality, designed into each of the run-time process executables, makes periodic 
system calls using the shell built-in function, times (prints accumulated process times for user and 
system), while its process is executing. Cumulative CPU time (as well as accumulated data regarding wall 
clock time and memory used by the process) is then output toa pstat file for each run-time process. Use 
of the ps command in the MSHN wrapper to compute user CPU time and system CPU time is discussed 
in Chapter IV, Section B. 
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correlated to wall clock time. It became clear that, in order to provide as controlled an 
environment as possible in measuring resource usage, the number of page faults would 
have to be standardized for each trial run. The only way to ensure this was to run each 
trial on a dedicated workstation, and at a time when there would be as little competition 
for memory as possible, thereby ensuring no page faults. Simulations were run 
consecutively, after normal working hours, on the SUN Ultra-1 workstation, and each 
trial was checked to ensure the absence of page faults. In reality, it is anticipated that 
EADSIM would be used by the warfighter in a “reach-back” situation. That is, the 
commander or planner would be forward deployed in a possibly computer-resource-poor 
environment, and would access EADSIM, located at a meta-computing center, via a 
reach-back network. Without the use of an RMS like MSHN, it is unlikely that the 
commander would know whether the server he was accessing was being shared by other 
applications or what impact this would have on his expected run time. It is sufficient to 
note, that wall clock time will increase if an application is required to swap more pages in 
and out of memory during execution. 

Using the Demo300 scenario, three runs of thirty trials each were conducted to 
gather the data needed to meet the experiment’s goals. The first run consisted of thirty 
Monte Carlo simulation trials executed by the wrapped version of each of the three 
EADSIM run-time processes (C3I, Detect, and FP). A run script (sh script) was written 
that allowed the seed for the model’s random number generator to be tied to the system 
clock (APPENDIX E). Both the C3I process and the Detect process were given different 
seeds for each trial (the FP process does not require a random number generator). Data 
collected from these thirty Monte Carlo simulation trials included MSHN wrapper output 
for each resource measured, cumulative CPU time as measured by EADSIM’s pstat 
facility, and post-processing reports that had been tailored to gather data on missile 
success or failure. The second run consisted of thirty deterministic (Planner Mode) 
simulation trials executed by the wrapped version of each of EADSIM’s three run-time 
processes. A separate run script similar to that used to launch the Monte Carlo simulation 
was employed, with no seed applied. Data collected from these thirty deterministic 
simulation trials included MSHN wrapper output for each parameter measured, 


cumulative CPU time as measured by EADSIM’s pstat facility, and a post-processing 
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report (the PAPA] report) that provided accumulated Pk data for each missile launched. 
Of note, only one such report was needed since simulation outcomes were deterministic. 
The final run consisted of thirty deterministic (Planner Mode) simulation trials executed 
by the unwrapped version of each of EADSIM’s three run-time processes. Data collected 
from these trials consisted only of cumulative CPU time as measured by EADSIM’s pstat 
facility. 

The data collected from the experiment trials were gathered into separate 
directories by another sh script, and copied into text files by PERL scripts written for this 
purpose. These text files were then copied into a statistical analysis computer program 
called Minitab which was used to produce descriptive statistics for each parameter 
measured. Minitab then produced histograms (see APPENDICES H and J) from these 
Statistics and plotted these histograms against normal curves for graphical comparison. 
As stated in Chapter I, Section C, follow on research will be conducted in an attempt to 
determine distributions for each of the resource usage parameters monitored by the 


MSHN wrapper [COOK99]. 


Cc. RESULTS FROM WRAPPED STOCHASTIC RUNS 


As discussed in Section B above, after executing preliminary runs to ensure 
required pages were cached in memory and no page faults were recorded, thirty Monte 
Carlo simulation trials of the Demo300 scenario were run using the three wrapped, 
EADSIM executables (C3I, Detect, FP). Output from the wrapper, the pstat facility 
and the post-processing reports was collected and stored in a dedicated directory. Data 


was sorted and, when appropriate, copied into Minitab for further analysis. 


1. Resource Measurements 


Computing resource usage data, collected using the MSHN wrapper during the 
thirty Monte Carlo simulation trials described in Section B, is summarized in Table 3. 
Graphic representations of statistics produced from those values that demonstrate 
variance during the Monte Carlo thirty trials are available in APPENDIX H. Values that 


did not change from one trial to the next are highlighted and noted with an asterisk. In 
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the FP executable, only once in the thirty Monte Carlo trials did the number of pages in 
the resident set vary (one trial had 7904 pages, vice the 7912 seen in the other Monte 
Carlo runs). This value is both highlighted and marked with a double asterisk. Since 
network reads (client machine seeking data from the network file server) were only 
necessary for process initialization, 1t was to be expected that the number of bytes read 
across the network, and the number of network reads, remained constant from one trial to 
the next for each process. Likewise terminal I/O, which was limited to output to the 
screen during the initialization process, did not vary from trial to trial. The number of 
data bytes read and the number of reads done locally (Local File Data) did not vary by 
process or from run to run (there were no local data writes recorded). This data was 
probably read from the local proc/ directory (related to the pstat functionality of 
each process). With the exception noted above, the number of pages in physical memory, 
virtual memory and resident set did not change from trial to trial in the FP and Detect 
processes, but did vary slightly from trial to trial in the C3I process (standard deviation 


over the thirty trials was minimal, as can be seen in APPENDIX H). 
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C3I Process FP Process Detect Process 


Local IPC: Number of Bytes 166758 3782 78411 
Read 


Local IPC: Number of Reads 20845 9801 









Local IPC: Number of Bytes 69328 2244479 
Written 


Local IPC: Number of Writes 1077 59372 


Local File Data: Number of | 2945 aye 
Bytes Read 





2945 


Local File Data: Number of 5 5 ye 
Reads a 






Network (NFS): Number of F086 704678% 
Bytes Read 

Network (NFS): Number of’ 4293 ro 
Reads 






Bytes Written 
Writes 


4] 
Terminal I/O: Bytes Written RHEE 530% 
Terminal I/O: Number of Writes 111s é 
System CPU Time 
User CPU Time 


Cumulative CPU Time 20.73 20.31 
(measured by EADSIM pstat) 















0% 
Nmber of Physical Pages in 2953 12817 29 
Memory 







Number of Pages in Virtual 23628 
Memory 


Number of Pages in Resident Set 17965 


* Outcomes did not change from trial to trial 
** Outcome of only one trial differed from the other 29 


Table 3: Mean Resource Usage, Over 30 Monte Carlo Trials, As Measured by 
MSHN Wrapper. 







2. Observed Simulation Outcomes 


Using the C3ISIM GUI interface on the SGI Indy workstation, we were able to 
tailor several post-processing reports to analyze the outputs from the stochastic runs of 
the Demo300 scenario. Having selected the observed number of times that at least one 
missile reached its target as our measure of effectiveness (Chapter V, Section C discusses 
the measure of effectiveness in detail), we tailored an Action History report to produce 


Red successes only (Figure 6.2). By simply counting the number of successes against 
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the missiles targets, as defined by the Demo300 scenario, we were able to determine the 
observed percentage of at least one missile reaching its target over the thirty Monte Carlo 
runs. Since there were seven occasions, in thirty trials, when at least one missile 


successfully destroyed its target, the proportion observed was 0.23. 


Engagement Statistics 


eecceee et ee Beene nw eee ee 


Scenario: Demo300 


Begin Report Time: 0 


{ 

! 

{ 

! 

{ 

! Report Type: Action History 
4 

' 

! End Report Time: 960 
' 

' 


Actions: 
Po ESOS 
! Hit_Base 
! Success 
/ 
! Acting A gainst 
! Platforms: Platforms: 
UO SSSRSRSSSEES 000m BRS SSK 
! All Hostile All Friendly 
! All Missiles All Ground Units 
! All Air Units 
! All M issiles 
! 
! Acting Against 
! Time Platforms Action Platforms 
LU SEBO SOOO BOSCO OO DOOR OR = EOSGDOCOOCOSDOOECOEEBOOOODO SOO SemCOCECOSEeeEHosGo6 
239.00 TBM _Depr_Traj_02 Hit_Base BASE 
246.42 Hostile _AA_Ftr_02/01 Success AA _Ftr_04/02 
294.50 Hostile _AA_Ftr_01/04 Success AA_Ftr_05/02 
301.02 Hostile_AA_Ftr_03/01 Success AA_Ftr_04/01 


Total Number of Actions: 4 


Figure 6.2 Sample Red Action History Report Summarizing All Red Missile Hits 
and Air-to-Air Combat Successes Against Blue Assets. After [EADS98] 


D. RESULTS FROM WRAPPED DETERMINISTIC RUNS 
1. Resource Measurements 


Resource usage data collected by the MSHN wrapper during the thirty, 
deterministic (Planner Mode) trials is summarized in Table-4. Since outcomes from a 
deterministic model do not vary from run to run, the only resource usage data that 
produced a non-constant distribution across the thirty tnals were user CPU time, system 
CPU time, and wall clock time. The resource usage values that did not vary from those 
observed during the Monte Carlo tnals are highlighted in Table 4. 

As expected, since the Planner Mode allows each missile to reach its intended 


target, and accumulates Pk data rather than allowing an engagement to result in a 
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platform being destroyed, each trial in the Planner Mode required more resources than 
trials run with the stochastic model. There was substantially more CPU usage and wall 
clock run-time in the Planner Mode, as well as more bytes written to disk, number of 
writes to disk, number of bytes read and written via interprocess communication, and 
number of IPC reads and writes. In the deterministic simulation (Planner Mode) only 
wall clock time, and system and user CPU (cumulative) time varied from one run to 
another. However, none of these increases would cause the expected run-time of a single 
deterministic run to be equivalent to thirty Monte Carlo trials. Graphical representations 
of statistics produced from these values that varied during the thirty deterministic trials is 
available in APPENDIX I. 


OS) B deere FP Process Detect Process 


Local IPC: Number of Bytes 223784 111768 
Read 


Local IPC: Number of Reads 27973 13971: 


Local IPC: Number of Bytes 119840 3098268 2497276 
Written 


Local IPC: Number of Writes 


Local File Data: Number of 
Bytes Read 

Local File Data: Number of 
Reads 

Network (NFS): Number of 
Bytes Read 

Network (NFS): Number of 
Reads 

Network (NFS): Number of 
Bytes Written 

Network (NFS): Number of 
Writes 

Cumulative CPU Time 
(measured by EADSIM pstat) 


Wall Clock Time 141.02 124.50 


Number of Physical Pages in 
Memory 


Number of Pages in Virtual 
Memo 


Number of Pages in Resident Set 


* Outcomes did not vary from those observed in Monte Carlo trials. 


Table 4: Mean Resource Usage for 30 Deterministic Trials, as Measured by MSHN 
Wrapper. 





68 


2: Computed Outcome Probabilities 


As discussed in Section B of this chapter, EADSIM’s post-processing GUI offers 
a series of reports useful for analyzing scenarios run in the Planner Mode. These reports 
are called “PAPA” reports, an important one of which is known as the PAPAI report. 
Although the PAPAI report provides analysts with a variety of information regarding 
hostile and friendly platforms, as discussed in Chapter V, Section C, we are most 
interested in the route probability of kill (route Pk) for each tactical missile (TBM and 
cruise) launched. We can obtain this data by scanning the PAPAI report for the last 
given route Pk for each missile. APPENDIX J provides a PAPAI report, with final route 
Pk’s of each missile highlighted. 

In Chapter V, Section C it was determined that our measure of effectiveness 
would require us to determine the probability that at least one missile reached its target (it 
is assumed in our experiment that if a missile is not destroyed, it will hit its intended 
target), in order to compare this probability with the outcomes observed in the Monte 
Carlo trials. Recall that the equation to determine whether at least one missile reached its 
target was: 


I] 2m; cfinat route Pky = p(all missiles are destroyed) 


il 


p(at least one missile is not destroyed) = (1 - p(all missiles are destroyed)) 
m = probability of kill for the i tactical missile (cruise missile or TBM) 
nm = number of missiles in the scenario 


From this PAPAI report, we find that there were a total of twenty-one TBMs and four 
cruise missiles launched, for a total of twenty-five tactical missiles. Taking the product 
of each missile’s final route Pk (highlighted in APPENDIX J), we find that the 
probability that all missiles were destroyed before hitting their target is 0.289. 
Subtracting this number from one, we conclude that the probability that at least one 


missile was not destroyed (and hence hit its target) is 0.711. 
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E. MEASURING THE OVERHEAD OF THE WRAPPERS 


In order to determine the overhead incurred by the MSHN wrapper, we ran thirty 
deterministic simulation trials of the Demo300 scenario using the three wrapped 
executable processes (C3I, FP, and Detect) and thirty trials using unwrapped processes. 
Using output from the pstat function discussed in Section B of this chapter, we found 
the mean cumulative CPU time of each process for each set of thirty trials. By 
comparing the mean cumulative CPU time gathered from the trials executed by the 
wrapped processes with the cumulative mean gathered from the trials executed by the 
unwrapped processes, we were able to determine the difference in average cumulative 
time for each process (Table 5). Since all trials were deterministic and run on the same 
workstation with the same file server, we attributed this difference to the overhead 
incurred by the wrapper. 

PSTAT DATA Cumulative CPU Cumulative CPU Difference 
(in seconds) Time Wrapped Time Unwrapped 


C3I Process 39.39 | 





Table 5: Mean Cumulative CPU Times Reported by PSTAT Function (Planner 
Mode). 


It is worth noting that the cumulative CPU times for the wrapped processes 
produced by the pstat function called by EADSIM closely agree with the cumulative 
CPU times measured by the MSHN wrapper. The mean cumulative CPU times (mean 
user CPU + mean system CPU) for the thirty deterministic simulation trials executed by 
the wrapped processes, as measured by the MSHN wrapper, are given in Table 6. In 
each case, the mean cumulative CPU time derived from data gathered by the MSHN 
wrapper was within approximately 1% of the mean cumulative CPU time derived from 
the pstat reports. As is evident from Table 5, only the Detect process incurred 
significant overhead (31%) from the MSHN wrapper. A graphical representation of the 


statistics derived from the wrapped and unwrapped executables running the Demo300 
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scenario in deterministic simulation trials, as measured by the pstat functionality of 
each process, is provided in APPENDIX K. 
MSHN Wrapper Data (in seconds) Cumulative CPU Time 











C3]I Process 39.43 
8s 


Table 6: Mean Cumulative CPU Times Reported by MSHN Wrapper. 
Since it was anticipated that the MSHN wrapper would add an insignificant 





amount of overhead to each wrapped process, it was surprising that the wrapped version 
of the Detect process demonstrated a nine second increase in cumulative CPU time over 
the cumulative CPU time measured for the unwrapped version. To attempt to determine 
the cause of this increase we took an algebraic approach that compared wrapper 
invocation overhead for the deterministic simulation trials. We assumed that any 
overhead added by the MSHN wrapper depended upon the actions taken by the wrapper 
upon intercepting a system call. The first step of this evaluation was to determine which 
wrapper calls did not vary from process to process, and which could not have contributed 
to the overhead incurred by the wrapper. Referring to Table 4, we noted that local file 
data and the number of page faults did not vary from process to process, and could 
therefore be eliminated as a possible source for the increased overhead displayed by the 
Detect process. Upon examining the code in APPENDIX C, we saw that the amount of 
data (number of bytes) transferred after a system call has been intercepted does not 
contribute to the overhead incurred by the wrapper. In other words, while the number of 
read(), write(), open(), close(), connect(), and accept( ) operations was of 
interest in evaluating the overhead accrued by the wrapper, the number of bytes 
transferred during, or as a result of, each of those operations would not have contributed 
to wrapper-generated overhead. By the same token, the number of pages in physical 
memory would not have contributed to the overhead created by the wrapper, since the 
wrapper simply gathers the data which is does by making a single, per process, 
invocation of the ps facility (Chapter IV, Section B discusses use of the Unix ps 


command, and code for the get PsData() function is available in APPENDIX D). 
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One final observation needs to be made before continuing our analysis. Terminal 
I/O debug statements, which were inserted in the MSHN wrapper source code during the 
debug process!°, were left in the code throughout each of the three, thirty-trial runs. 
When the experiment was completed, we removed the debug statements and executed the 
Demo300 scenario using the wrapped executables running the deterministic (Planner 
Mode) simulation. We found that all twelve of the terminal I/O writes for the Detect 
process noted in Table 4 had been eliminated, as had five of the one hundred and eleven 
terminal I/O writes performed by the C3] process and thirteen of the twenty-one terminal 
I/O writes performed by the FP process. While this would explain part of the total 
overhead incurred by the Detect process, since the FP process had more terminal I/O 
attnbuted to the debug statements than did the Detect process, we concluded that the 
debug statements can also be eliminated as a source of the added overhead observed in 
the Detect process. 

In comparing the values measured for each of the three processes, one value 
stands out as being significantly different for the Detect process than for either the C31 
process or the FP process. The ratio of Detect’s system CPU time to its cumulative CPU 
time is significantly greater than either of the other two processes’ ratios of system CPU 
time to cumulative CPU time. This was observed in both the deterministic simulation 
trials and the stochastic simulation trials. Perhaps more interesting, however, is the fact 
that although the Detect process required far less user CPU time than the C3] process to 
execute the Demo300 scenario in the deterministic simulation, the Detect process 
required more than twice as much system CPU time than did the C3I process (which was, 
in turn, more system CPU time than was required by the FP process). This seemed to be 
an excellent clue for evaluating why the MSHN wrapper incurred more overhead in the 
Detect process than in the other two processes. Something within the Detect source code 
was requiring that more time be spent in system calls than in the other two processes. We 
decided to compare the quantity of each different type of system call that the wrapped 
process made, in an attempt to determine which, if any, of Detect’s system calls were 


significantly greater in number than in the other two wrapped processes. 


10 #Ifdef MSHN DEBUG statements written into many of the MSHN client library functions were not 
defined for use during this research. Debug statements were instead inserted into four functions only. 


i. 


Again referring to Table 4, we concentrated our effort on the following values: 
number of local IPC reads and writes; number of local file data reads; number of remote 
file data read from and written to the network file server; and the number of terminal I/O 
writes. As can be seen by scanning Table 4, in no case did the Detect processes’ resource 
usage measured for any of these values exceed the usage measured for each of the other 
two processes (the number of reads and writes listed in the Detect Process column of 
Table 4, is never the maximum value of a given row). In all but one case (the number of 
IPC reads) both the C3I process and the FP process performed at least as many read and 
write Operations as the Detect process. And while the mean number of IPC reads is 
greater for the Detect process than the FP process, this number is far less than the mean 
of observed IPC reads for the C3] process. 

Based on the fact that the Detect process did not perform the most read or write 
system calls, of any given type (local IPC, local file data, remote file data, and terminal 
I/O), we eliminated wrappers of read and write system calls as the cause of the significant 
wrapper-induced overhead observed in the Detect process. However, as discussed above, 
we know that the Detect process did require significantly more system CPU time than the 
other two processes. To understand how this can be possible we reviewed the way in 
which EADSIM models a scenario (EADSIM is described in detail in Chapters III and V 
and the Demo300 scenario is described in Chapter V). 

Through IPC, the C3I process tells the Detect process which platforms are of 
interest. The Detect process, which has been initialized with sensor characteristic data 
for each of the scenario’s platforms, receives “ground truth” on the movement of these 
platforms via IPC from the FP process. The Detect process uses this data to compute 
sensor output, then periodically reports sensor information to the C3I process. While 
EADSIM’s source code was not used in this analysis, it was verified by TBE engineers 
that the Detect process awaits ground truth from the FP process, in order to make its 
periodic report of detections to the C3I process. We believe that during each time-step 
interval, the Detect process queries the FP process, in a loop, until ground truth data is 
sent. This is known as “busy waiting.” In this busy waiting situation, the Detect process 
would use the CPU to check (in a loop) whether the FP process had data to transmit. The 


MSHN wrapper does cause some overhead each time the Detect process checks the FP 
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socket for data) Even when no data is available in the socket, the wrapper calls 
getclocktime( ) twice. To eliminate the possibility that the MSHN wrapper was 
adding overhead by invoking the getclocktime( ) system call each time the Detect 
process checked for data from the FP process while busy waiting, the call to 
getclocktime( ) was commented out of the readWrapper( ) function in 
MSHN_syscall_lib.cc. This change, however, resulted in no discernable reduction in 
system, or cumulative, CPU time required to execute the Detect process. 

The MSHN wrapper does not currently report very fine grained data regarding 
system calls. In fact, no data is output regarding the number of open( ), close( ), 
connect( ), or accept( ) system calls. Therefore, having eliminated the data that the 
MSHN wrapper output to file as providing sufficient information from which to deduce 
the cause of the additional overhead incurred by the Detect process, we conclude that we 
currently have too little data to determine what caused the MSHN wrapper of the Detect 
process to incur more overhead than the wrappers of the other two processes. The 
solution to this problem will be discussed in the Future Work section of the next chapter. 

One last note regarding the data output to file by the MSHN wrapper. It is 
apparent in Table 4 that the total number of IPC writes do not equal the total number of 
IPC reads, nor do the total number of IPC bytes written equal the total number of IPC 
bytes read. It is unclear, based on the data available from the MSHN wrapper, why this is 
so. Work is currently underway to modify the MSHN wrappers so that finer grained data 


will be collected and reported. 


P. WEIGHING THE SIMULATION RESULTS USING OUR MEASURE OF 
EFFECTIVENESS 


As discussed in Chapter V, Section A, we have chosen EADSIM as being 
representative of a complex C4I modeling application that offers the warfighter a variety 
in Quality of Service. Based on the operational situation, the commander or mission 
planner, with the aid of an intelligent resource management system such as MSHN, can 
select that version of EADSIM that best suits his or her needs. This decision will take 
into consideration the time constraints, system security limitations, mission priority, and 
computing resource availability (e.g., network bandwidth and anticipated congestion, 


eo"? 
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local memory and disk space, graphic display capabilities). Chapter V, Section C, 
explains a measure of effectiveness that would allow us to weigh the results obtained 
from running EADSIM in a deterministic simulation with the results obtained from 
running EADSIM in a stochastic simulation. In other words, in a time constrained 
situation, given that the deterministic simulation, though more resource intensive than the 
stochastic simulation, need only be run once, can the results obtained from the 
deterministic version of EADSIM be useful? Or must mission planning be delayed 
sufficiently long to enable the Monte Carlo runs to be executed a large number of times? 

Table 7 displays the computed probability (from the Planner Mode run) that at 
least one missile would not be destroyed (and would hit its target) and the observed 
percentage of time (from the Monte Carlo runs) that at least one missile hit its target. 
What, if any, conclusions can be drawn from this data? 

Using data from the PAPA] report described in section D, subsection 2, and our 
equation to determine the probability that at least one missile hits its target, the 
commander who chose to run the deterministic simulation due to time constraints, would 
have been told there was a 71% chance, one of his or her bases would be hit by a missile. 
Had the same commander, with more time available to analyze the scenario, chosen to 
run thirty stochastic trials, he or she would have observed one of his or her bases hit by a 
missile in only 23 % of the runs. Although the deterministic simulation offers a 
significant savings in time, there was quite a disparity between the probability of a 
missile hit, and the observed proportion of times at least one missile hit its target. 


p(at least one missile hits its 


Deterministic Simulation 


Stochastic Simulation 


Proportion of trials when at 
least one missile hit target 


Table 7: Comparison of Predicted and Observed Outcomes. 





To determine whether the observed disparity could simply be due to chance, or 


the overall probability of a missile reaching its target in the stochastic trials was, if fact, 


75 


different than that for the deterministic trials, we next tested the hypothesis that the true 
underlying probability of at least one missile hitting its target during the stochastic trials 


Was p = ./1 1. Each run of the stochastic simulation can be considered a Bernoulli trial, 
with the probability of at least one missile hitting its target equal to p» and the probability 
of no missile hitting its target equal to I-p. The number of success (trials in which at 
least one missile hits its target) of , repeated independent Bernoulli trials (each with 
probability ,,) follows a Binomial distribution with parameters (, and p). In our case, the 
observed number of successes (in , = 30 trials) was 7. So, the objective of the test was to 


determine whether 7 successes in 30 trnals was a reasonable outcome if the true 
probability of success in each trial was .711. 

Figure 6. 3 provides the following: (i) the Binomial distribution and values for p, 
x, and p; (ii) the null hypothesis, that the probability of at least one missile hitting its 
target is .711; (111) the observed proportion of successes (at least one missile reaching its 
target) in thirty Monte Carlo trials; and (iv) the probability that an observed proportion of 
successes would be less than or equal to .233, given the probability of at least one missile 


hitting its target is .711. 


n 
(i) Binomial! Distribution: b(x; 7, p) = : P* (-py* x=0,1,...,7 


0 otherwise 
Where: 


n= number of tnals =30 
x= number of successes =7 


p= probability of success =.71 1 


(il) Null hypothesis (H,): p(at least one missile reaches its target) = .711 


ooo A 
(111) 2 =proportion of x/n = 7/30 =.233 
(iv) p(P <.233 |[p=.711)= 0.0000 


Figure 6. 3 Testing the Null Hypothesis that the Probability of at Least One Missile 
Reaching Its Target is .711. 
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As Figure 6. 3 part (iv) shows, if the true probability of at least one missile 
reaching its target is .711 (our null hypothesis), then the probability we would observe a 
result as unusual as x = 7 or less (the p-value), is essentially 0.0. Therefore we can reject 
the null hypothesis with any reasonable level of significance. In other words, applying a 
two-tailed hypothesis-test to determine whether our observed proportion of successes 
(success is defined as at least one missile reaching its target) supports our null hypothesis 
with an acceptable level of confidence, would lead us to reject the null hypothesis. By 
rejecting the null hypothesis, we have concluded that the overall probability of at least 
one missile reaching its target in thirty Monte Carlo trials is not equal to the .711 derived 
from the deterministic simulation. While rejection of the null hypothesis means that we 
cannot simply substitute one version of the model for the other, it does not infer that the 
deterministic version of EADSIM cannot be of use if time constraints (the operational 
deadline) eliminate the possibility of running the stochastic version. It would be very 
helpful, however, to know which direction the disparity might go, and to have an estimate 
of the bias. 

The defending commander, using EADSIM, who chose to go with the 
deterministic version of the model, would know that he or she had received a worst case 
predicted loss. Planning based on this anticipated result, may have been far more 
conservative than necessary, but in wartime this may not be a bad alternative. As long 
as the commander was apprised of the potential for gross over-estimation of the enemy’s 
capabilities, the result from the deterministic simulation may represent a “better than 
nothing” estimate. Furthermore, it must be remembered, that the PAPAI report 
(APPENDIX J) is a valuable resource for analysts, providing a means to follow scripted 
platforms from launch to intended target, accumulating probability of kill and flight 
geometry data along the route. The defending commander for instance, might 
concentrate more defensive counter air in the area of the Red Cruise Missile Number 2, 
which, according to the PAPA] report, had only a 70% chance of being destroyed before 
it reached its target (vice over 97%, for all other cruise missiles). 

On the other hand, had the offensive planners using EADSIM decided they only 
had time for the deterministic version of the model, they would know that they had 


received an overly optimistic probability that at least one of their missiles hits its target. 


ia 


But the computed probability of at least one missile hitting its target was just the measure 
of effectiveness chosen for this experiment. The PAPA] report would have provided the 
offensive planners with valuable data regarding the relative probability of kill for each of 
their platforms for each leg of the route to their intended targets. This information might 
be used to evaluate strike packages, flight routes, target accessibility, sensor 
requirements, or placement of ground assets. In some after-action analysis, contingency 
and mission planning, data gathered from running scenarios in the Planner Mode 
(deterministic simulation) may be just as valuable as running stochastic simulations for 
observed outcomes. [t must be stressed that QoS is determined by the user, in our case 
the warfighter, who will choose the version of an application that most closely suits the 


needs and constraints of the given operational situation, based upon advice from an RMS. 


G. SUMMARY 


This chapter described the experiment that allowed us to compare the resources 
required to execute EADSIM running both stochastic and deterministic simulations, to 
compare the results obtained from these two configurations of EADSIM, and to 
determine the overhead incurred by the MSHN wrapper. It provided the suggested 
minimum hardware requirements to run the EADSIM application and described the 
computing environment in which our experiment was conducted. It detailed the 
methodology we used to obtain data in support of the experiment’s objectives. It 
reported the resource usage derived from the thirty stochastic simulation trials and the 
observed simulation outcomes and the resource usage from the thirty, deterministic 
simulation trials that were run using the wrapped executables. It analyzed, as best as 
possible, the observed overhead attributed to the MSHN wrapper. Finally this chapter 
weighed the results obtained from the deterministic simulation against the results 


obtained from the stochastic simulation. 
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VU. CONCLUSIONS AND FUTURE WORK 


This thesis presented the methodology of intercepting, or wrapping, system calls 
made by the Extended Air Defense Simulation (EADSIM), a robust C4I, air and missile 
warfare modeling application, in order to determine the resources required to execute the 
program on a stand-alone workstation. Having demonstrated the ability to measure the 
application’s resource usage without requiring access to source code, an experiment was 
described in which the resource usage was measured running the application in both 
Monte Carlo simulations and deterministic simulations. The outcomes obtained from 
running EADSIM in both deterministic and stochastic simulations were then weighed 
against each other. This chapter will discuss the contributions of this thesis and future 


work. 


A. CONCLUSION 


The goal of MSHN is to provide a resource management system (RMS) that will 
enable adaptive applications to provide each subscriber process with its required Quality 
of Service (which might include security considerations, deadlines, user priorities, and 
preferences) in a distributed, heterogeneous computing environment in which many 
processes are competing for shared resources. The MSHN architecture is designed as a 
system that is capable of being integrated with, and incorporating, a variety of distributed 
system tools (for example, CORBA, ENSEMBLE, and COMPASS) to reap the 
maximum benefits from available resources. 

In a military environment, where the use of distributed, and _ possibly 
heterogeneous, systems represents both challenge and opportunity, the MSHN RMS 
would allow a user to select the most appropriate application, or version of an 
application, capable of executing within a specified time, at the proper security level, in 
order to deliver the most complete answer achievable within stated time constraints. 
Applying this technology to C4I modeling and simulation applications would enable on- 
scene commanders and mission planners to simulate complex elements of the decision 


process in order to optimize the use of forces and materiel. As discussed in Chapter II, 


We: 


critical to this implementation 1s the development of adaptive and adaptation-aware 
models and decision support applications that exist in different versions, designed to 
satisfy user-defined Quality of Service (QoS) parameters. 

It was explained in Chapter JJ that adaptive applications exist in different versions 
capable of producing like results (though possibly offering varying degrees of QoS). 
MSHN would monitor the use of such adaptive applications and would be able to 
terminate one version and start another, possibly from the beginning, if it perceived the 
user's QoS requirements were not being met by the currently executing version. In a 
tactical environment, this means that the transition to improved QoS recommended by the 
MSHN RMS, if accepted by the decision-maker, would transparently enhance his or her 
mission effectiveness while remaining within given time constraints. 

Chapter IV described the experiment that we conducted to weigh model results 
against the resources required to execute deterministic and stochastic model simulations. 
This experiment demonstrated MSHN’s ability to measure an application’s resource 
usage without requiring source code. The overhead associated with the MSHN wrapper 
was measured and, in Chapter VI, possible causes of this overhead were discussed. The 
experiment led to the realization that the MSHN wrapper needs to be modified to collect 
finer grained information, specifically, send ( ), sendto ( ), sendmsg ( ), recv ( ), 
recvfrom(), recvmsg(), select (), and listen () system calls may need to be 
wrapped. Additionally, more information may be required from the current wrapper. 
Chapter VI also used the Binomial distribution to help evaluate the trade-off between the 


fidelity of results from EADSIM, a sophisticated C4] simulation. 


B. FUTURE WORK 
1. Development of a MSHN Application Emulator 


Resource usage data gathered by the MSHN wrapper will be used for, among 
other things, the MSHN Application Emulator (AE). The AE will produce predictive 
Statistics regarding resource requirements by simulating the running of applications 
without the accompanying overhead. The AE will also be used to monitor the status of 
resources not being used by MSHN applications. In order to simulate an application, the 


AE must know that application’s resource requirements and the probability distribution(s) 
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of those requirements. Data from applications wrapped by the MSHN Client Library, 
similar to the data reported for EADSIM resource usage, will be used in the MSHN AE, 
along with their appropnate distributions, to simulate applications being executed in a 


distributed environment. [DRAK99] 


Z. Dynamically Determining Distribution Statistics for Resources in 
a Distributed Environment 


Statistical analysis needs to be conducted, using resource usage data collected by 
the MSHN wrapper, to determine whether there is a family of distributions for a given 
resource requirement (similar to the Student’s t family of distributions) that can be 
adequately modeled by an underlying exponential distribution. Assuming such an 
exponential family of distributions exists, Monte Carlo simulations will be necessary to 
compare the performance of greedy and exhaustive scheduling algorithms using the 
exponential family of distributions with the performance of these same scheduling 
algorithms using a normal distnbution. Research will also have to determine what 
sample size is needed for the resultant distribution to be within an acceptable level of 


significance of the true exponential distribution. [COOK99] 


3. Refining a Model for Use in Scheduling in MSHN 


In order to accurately predict an application’s performance, in a given distnbuted 
environment, a network system model will need to be developed. This model will use the 
MSHN Application Emulator to emulate computationally-intensive and communication- 
intensive, synchronous and asynchronous, applications. Using data generated by the 
MSHN wrapper and the Application Emulator, the network system model will focus on 
predicting run-times for communication-intensive and computationally-intensive 
processes. Since MSHN’s goal 1s to provide a resource management system (RMS) that 
will provide each subscnber process, when possible, with its required Quality of Service 
in a distnbuted, heterogeneous computing environment, this information 1s needed to 
determine whether next-generation C4I requirements (among others) can be supported by 


Commercial-Off-the-Shelf and Government-Off-the-Shelf products. [SHAE00] 


8] 


4. Testing Resource Monitoring Tools on a Win32/Intel Platform 


As part of Information Technology, 21* century (IT21) the US Navy has made a 
commitment to transition from UNIX workstations to a Windows-based, NT computing 
environment. Methods of monitoring resource usage for applications being executed on 
Win32x86, or NT, platforms are needed. Such monitoring will parallel the methods used 
by the MSHN wrapper in the UNIX computing environment. In other words, a method 
of measuring resource usage for an application run on NT workstations and servers will 
be sought that does not require access to the application’s source code. It is anticipated 
that this investigation will include work done with the Executable Editing Library (EEL) 
developed by James Larus of the University of Wisconsin, and an extension of EEL for 
the Win32 Platform developed by the Etch team from the University of Washington. 
[CHENOO] 


5. Expansion of Existing MSHN Wrapper Functionality 


As discussed in Chapter VI, the MSHN wrapper will need to be expanded to 
capture more fine grained data than is currently being measured. This will mean that 
several more system calls will need to be intercepted, allowing more resource data to be 
analyzed. System calls that need to be wrapped include send ( ), sendto ( ), 
sendmsg (), recv (), recvfrom(), recvmsg (), select (), and listen ( ). 
Once these system calls have been wrapped by MSHN, further experiments can be run on 
EADSIM and other military applications, to better understand resource requirements, and 


to facilitate the research discussed in Section B, Subsections 1-4 above. 
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APPENDIX A. MODIFIED README-FIRST FILE 


This README FIRST file was written by Matthew Schnaidt for use with the 
MSHN Chient Library files [SCHN98]. I have modified this file for use with the Solaris 


2.5 (Sun5) operating system (changes noted in bold). 


Phe ene OME Pik ST, 


1. PURPOSE. The purpose of this readme file is to assist the user in 
using the MSHN client library. A certain level of understanding of the 
client library is assumed. Matt Schnaidt’s December ‘98 thesis is the 
basis for this knowledge. Specifically, Chapters 2, 4, 5, 6, 7 and 
Appendices B and C. The author strongly recommends working through the 
tutorial in Appendix C prior to trying to run link with the client 
library. 


2. SUBDIRECTORIES. The subdirectories in this directory contain files 
used by the client library as well as test programs used with the 
client library. The subdirectories and the purpose of each are 
enumerated below. 


/syscall_1lib This directory contains the files required to 
wrap system calls. 


fextract This directory contains the makefile that 
creates the modified C library used by the syscall library wrappers. 


/makeUppercase This directory contains the code for the 
makeUppercase.cc and makeLowercase.cc programs. 


Jelvock This directory contains the code for the 
clockServer used in estimating clock offsets. 


/thesisTutorial This directory contains the code presented in 
Appendix C of Matt Schnaidt’s thesis. This presents a simple tutorial 
on how to wrap system calls. 


/socketTest This directory contains a client and server 
program. These programs are wrapped with the client library. A server 
is started on one machine and a client is started on another; these 
programs simply transfer a sample file back and forth in order to 
demonstrate the client library’s functionality. 


/testoverhead This directory contains code for testing the 
overhead incurred by the client library. 


3. RUNNING A WRAPPED PROGRAM. Fach subdirectory has a README file that 
explains how to use it. In order to wrap an application, the following 
is a high level description of tasks that must be accomplished: 


a. Create the modified C library, libMSHNc.a. Enter the /extract 


directory, and run the mshnlibc*Makefile (where * is the current OS 
name - Linux, Sun4 etc) (e.g., make -f mshnlibcSun4Makefile). This 
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will create the modified C library, libMSHNc.a. Copy this library into 
the /syscall_lib directory. 


b. Create the client library, MSHN_syscall_lib.o. Enter the 
/syscall_lib directory (ensure that you’ve copied libMSHNc.a to this 


directory). Check the file MSHN_types.h to ensure that the correct 
variables are defined (e.g., LINUX, SUN4, SUN5), and then compile using 
the makefile, libraryMakefile(e.g., make -f libraryMakefile). At this 


point, you now have the client library, MSHN_syscall_lib.o. 


c. Modify the C Run-Time object file. Enter the extract directory. 
Copy the C run-time object file to this directory and rename it 
Mod_crt*#o (on Sun4 this 1s crtQ@ vom so cp /usr/ lib/ercl co Medsc > eum 
in Solaris 2.5 this is crtl.o, so cp /usr/lib/crt0.o ./Mod_crtl.o). 
Now, modify the C run-time object file’s reference to main() 
(makeUppercase Mod_crt0.o main or Mod_ertl.o main). In order to link 
any applications with this one, you will link with MSHN_syscall_lib.o. 
Steps a, b,and c need not be repeated. 


d. Link your application with the client library and modified C run- 
time object file. First, copy the MSHN_syscall_lib.o and Mod_crt0.o 
(or Mod_ecrti.o) files into this directory. The simplest way to do this 
is to compile your application into a single, non-executable, object 
file (e.g., myApplication.o). Then linking will take place in two 
phases: preparation, and linking. We are going to modify the link 
command generated by the compiler. So, the first step is to determine 
the link command that the compiler uses, and to modify it to replace 
the system’s C run-time file with our own. On the Sun4 machines, we 
used the "-v" (verbose) flag with our link command: 

g++ -v myApplication.o MSHN_syscall_lib.o -o myApp. 


This generates the following output to the screen: 


/usr/local/lib/gcec-lib/sparc-sun-sunos4.1/2.6.3/ld -e start 
-dc -dp -o myApp /lib/crt0O.o -L/usr/local/1lib/gcc-1lib/sparce-sun- 
sunos4.1/2.6.3 -L/usr/local/12b myApplicatton.o MSHN syvscaliet > 76. 
lg++ -lgcec -le -lgcc 


We then replace the system’s default link "/lib/crt0.o" (or 
"/lib/ert1.o") with "./MSHN_crt0.o" (or “"./MSHN_crti1.o") and use this, 
as our second step, to generate our executable. 


/usr/local/lib/gec-lib/sparc-sun-sunos4.1/2.6.3/ld -e start -dce -dp -o 
myApp ./MSHN_crt0.o -L/usr/local/1ib/gcec-1ib/sparc-sun-sunos4.1/2.6.3 - 
L/usr/local/lib myApplication.o MSHN_syscall_lib.o -lg++ -lgcc -lec - 
leee 


or 
/usr/local/1lib/gcc-1ib/sparc-sun-sunos4.1/2.6.3/1ld -e start -dce -dp -o 
myApp ./MSHN_ecrtl.o -L/usr/local/lib/gec-1lib/sparc-sun-sunos4.1/2.6.3 - 


L/usr/local/lib myApplication.o MSHN_syscall_lib.o -lg++ -lgcce -lc - 
Lace 


e. Compile and start the clock server. Enter the clock directory, 
compile clockServer.cc and then run clockServer. clockServer listens 
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at a fixed port (#12391) defined in clockIncludes.h. Additionally, a 
variable is defined in clockIncludes.h if the server runs on a LINUX 


platiorm. 


f. Run the application. 
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APPENDIX B. MSHN LIBRARY MAKEFILE2 


Makefile for MSHN Client Library, modified to explicitly link with c++ and g++ 
compilers, the math and c libraries. 


CC= G++ 
HiniGomnenii nom SUNS  CC=CGe 


MSHN_syscall_lib.o: MSHN_monitor_RRD_Class.o MSHN_utility.o 
MSHN_MAINc.o\ 

hashClass.o MSHN_MAIN.o MSHN_network_IO.o 
MSHN_export_RSS_Class.o 

S{CC} -03 -c MSHN_ syscall_lib.cec -o 
MSHN_syscall_temp.o 
# ld -i -g MSHN_syscall_temp.o MSHN_monitor_RRD Class.o 
MoM muted lity oO 5 \ 
hashClass.o MSHN_MAIN.o MSHN network_I0O.o 
MSHN_export_RSS_Class.o \ 
# -L../ -1MSHNc -o MSHN_syscall_lib.o 

ld -r -t -Bstatic MSHN_syscall_temp.o 
MSHN monitor RRD Class.o \ 

MSHN_utility.o hashClass.o MSHN_MAIN.o 
MSHN_network_IO.o MSHN_MAINc.o \ 

MSHN_export_RSS_Class.o -L./ -1MSHNc -L /usr/local/1lib 
-lg++ -lstdc++ -lm -lc -o MSHN_syscall_lib.o 

rm MSHN_syscall_temp.o 

rm MSHN_utility.o 

rm MSHN_monitor_RRD_Class.o 

rm hashClass.o 

rm MSHN_MAIN.o 

rm MSHN_export_RSS_Class.o 

rm MSHN_ network_IO.o 

rm MSHN_ MAINc.o 


MSHN monitor _RRD Class.o: MSHN_monitor_RRD_ Class.cc 
MSHNeuta lrty fh \ 

MSHN_types.h 

S{CC} -O3 -c MSHN_monitor_RRD_Class.cc 


MSHN_MAIN.o: MSHN_MAIN.cc 
S{CC} -0O3 -c MSHN_MAIN.cc -o MSHN_ MAIN.o 


MSHN MAINc.o: MSHN_MAINc.c 
gcc -c MSHN_MAINc.c -o MSHN_MAINc.o 
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MSHN_ network_IO.0o: MSHN_network_IO.cc 
S{CC} =-0O38>—-e MSHN network Merce 


MSHN_utility.o: MSHN_utility.cc MSHN_types.h 
S{CC} =-03 -—ec MSHN Utility -ce 


hashClass.o: hashClass.cc 
S${CcCC} -O3 -c hashClass.cc 


MSHN_export_RSS_Class.o: MSHN_export_RSS_ Class.cc 
S${CC} -O3 -c MSHN_export_RSS_Class.cc 
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APPENDIX C. MSHN_SYSCALL_LIB.CC 


a a RE KR KE EE eee EE 


// File: MSHN_syscall_lib.cce 

// Name: Matt Schnaidt (modified with permission by W. Porter 
// Operating Environment: Linux 2.0.29, SUN OS 

// Compiler: g++ for Linux, Unix 

// Last Modified: 7 Feb 99 

ia 


// Description: When this file is included in another program, it 
Te causes all calls from that program to read, write or exit, to 
ss be "caught", information 

Gi recorded, and passed on to the operating system. 

// Inputs: The operating system calls. 

/7 OvUEDUES: none: 

// Process: none 


Calling progams input proper data type. 
This warning cannot be 


// Assumptions: 
// Warnings: no return function returns. 


vi gotten rid of without major work-arounds. 

Pie nt 2 eh OM st tt 
ef 

#include <stdio.h> 

#include <iostream.h> 

#include <sys/types.h> 

#include <netdb.h> //for gethostent in accept 
#include <strings.h> 

#include "MSHN_syscall_lib.h" 

#include "MSHN_monitor_RRD_Class.h" 

#include "MSHN_export_RSS_Class.h" 

#include "MSHN_utility.h" 

#include "MSHN_types.h" 

#include "hashClass.h" 

#inelude “clockincludes: h* 

#include "MSHN network_IO.h" 

//OBJECT declarations 


//the object that tracks all resource usage 
extern MSHN_monitor_RRD_ Class resourceMonitor; 
MSHN_export_RSS_Class rssObject; 


//the object that keeps track of what the type is of each fd 
hashClass fdTable; 


//define prototypes for wrapper functions 
vVOld e€xitwrapper(int, vorvd-) ane) 

int readWrapper(int, char*, int, int(*) (int, 
int. weitewrapper (int, chars. ime, ime (*) (ime, 
int closeWrapper(int, int(*) (int)); 


AiG )a) 5 
Beil) ) 


Ghar, 
enar | 


int openWrapper (const Char*, intfeinmt, ine(*) (constvwenar*, Int ame) ) - 


int soecketWrapper (int, invemene, tne (*) (ine, ant: int) }-; 
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Int acceptwrapper(int, Struct sockaddr ine 
int(~) (int, struct sockaddr 1c o. 
int connect Wrapper (int, struct sockaddr*7antc 


int (* (int, StErUcE SoOckadae. 7) i1me ine 


//declare the redefined clib symbols as available externally 
extern "C"{ 

//these must be defined for LINUX 

#ifdef LINUX 

extern int ._ READ(ant, char”, int); 

extern int ~ WRITE((nt, char) ee). 

extern int __OPEN(const char*, int, int); 

exterm int 9 CLOSE (ant): 

#endif 


//these must be defined for sun 5.5 

#ifdef SUN55 

G@xtern Int - READ (Ine, eecmar” ene). 

extern Ant “WRETE(@nE, Chan .. ant) 

extern int _OPEN(const char*, int,int); 

extern int _CLOSE(int) ; 

extern int _SOCKET(int, int, int); 

extern int _ACCEPT (int, struct sockaddr*, int*); 
extern int _ ACCEPT(int, struct sockaddr*, int*); 
extern int _CONNECT(int, struct sockaddr*, int); 
extern int —CONNECT2 (int, struct sockaddr: ne). 
#endif 


//these are required for Linux, Sun 4.x, Sun 5.x 

extern int READ(int, char*, 2mt)-; 

extern ane WRITE (int, (ehar ; ame) 

extern int OPEN(cOnsE Ghar’ seine ine) 

extern int CLOSE(int) ; 

extern void EXIT(int); 

extern void _EXIT(int); 

extern ant SOCKET (int, 2nt,, 1nt): 

extern int LISTEN(int, int); 

extern int ACCEPT (int, Struct soeckadds-,1nt )- 

extern int CONNECT (int, struct sockaddr) ine) 

extern int SEND(int, const void*, int, unsigned int); 

extern int SENDTO(int, consStwvo1d*, int) unsigned int, 

const Struct. soekadd: 7 ime), 

extern int SENDMSG(int, const struct msghdr*, unsigned int); 

extern int RECV(int, void*, int, unsigned inet) ; 

extern int RECVFROM(int, void*, int, unsigned int, struct sockadadews 
Int*)= 

extern int RECVMSG(int, struct msghdr*, unsigned int); 
}//end extern "C" 
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// Function: void exitWrapper () 


// Purpose: Outputs to the RRD this application’s name and 


7 resource utilization, then calls the system’s exit 

(Op EUNGir lon 

A a 
Ve 


void exitWrapper(int status, void(*exitFunction) (int) ) 


{ 
#ifdef MSHN_DEBUG 
printf("--inside exitWrapper--\n"); 
#endif 


resourceMonitor.updateResourceServer (status); 


//pass the system call on 
(*exitFunction) (status) ; 


//will not return 
}//end exitWrapper 


// Function: readWrapper () 


// Purpose: Calls the system’s read system call, calls the 


17 resource monitor which updates number of reads 
ei and number of bytes read. 
fie ee 


int readWrapper(int fd, char* buf, int len, 
int (*readFunction tint, chars, Int) ) 


{ 


j7/used to Measure -auraeren wor reads 
Struct timeval stare@@iime, 
endTime; 


oi 


int returnValue OF 


tempValue = 0; 
double readDuration; 


bucketElement* fdPtr = fdTable.lookUp(fqd) ; 
fd_type thisFdType; 


Jj/2f ptr 1s null, ‘nen thisvis stds. 
Wm (CaPte) { 
thisFdType = fdPtr->type; 
} 
else{ 
thisFdType = TERMINAL_I0O; 
}//end if 


#ifdef MSHN_DEBUG 
cout<<"inside readWrapper, fd => "<<fd<<", FdType=> 
"<<thisFdType<<endl; 
#endif 


if (thisFdType==LOCAL_FILE) { 


//start the clock, make the system call, and stop the clock 
returnValue = (*readFunction) (fd, buf, len); 


//update the read counters 

resourceMonitor.updateLocReads (returnValue) ; 
} 
//if this is a read from across the network, check latency and b/w 
else if(thisFdType == NETWORK_IPC) { 


//start the clock, make the system call, and stop the clock 
startTime = getClockTime(); 
returnValue = networkRead(fd, buf, len, startTime, 


fdaPtr, readFunction); 
endTime = getClockTime(); 


//update the terminal read/write information 
double elapsedTime =(calcTimeDiff(&startTime, &endTime) ); 


//update the network reads 
resourceMonitor.updateNetReads(returnValue, elapsedTime) ; 
} 
else if(thisFdType == LOCAL_IPC) { 
returnValue = (*readFunction) (fd, buf, len); 
resourceMonitor.updateLocIPCReads (returnValue) ; 
} 
//if this is a file accessed over the network, collect access 
// time data 


else if (thisFdType == TERMINAL_IO) { 
//start the clock, make the system call, and stop the clock 
startTime = getClockTime (); 
returnValue = (*readFunction) (fd, buf, len); 
endTime = cetClockTime (> 
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//calculate the time it took to do the read 

double elapsedTime =(calcTimeDiff(&startTime, &endTime) ); 
//update the read counters 
resourceMonitor.updateTerminalReads(returnValue, elapsedTime) ; 


} 


else if (thisFdType == REMOTE_FILE) { 
//start the clock, make the system call, and stop the clock 
startTime = getClockTime(); 
returnValue = (*readFunction) (fd, buf, len); 
endTime = getClockTime(); 


//calculate the time it took to do the read 
double elapsedTime =(calcTimeDiff(&startTime, &endTime) ) ; 
resourceMonitor.updateDistReads(returnValue, elapsedTime) ; 


} 


else{ 
returnValue = (*readFunction) (fd, buf, len); 
resourceMonitor.updateLocIPCReads (returnValue) ; 

}//end if 

#ifdef MSHN DEBUG 
cout<<"--about to leave read wrapper,read # bytes "<<returnValue; 
cout<<" --"<<endl<<endl; 

#endif 


J/ceturns the Size of Ehe read 
return (returnValue) ; 


}//end readWrapper () 


ee ee ee ee ee ee 


// Function: writeWrapper () 


// Purpose: Calls the system’s write system call, calls the 


resource monitor which updates number of writes 


and number of bytes written. 


ed 


int writeWrapper(int fd, char* buf, int len, 


int (*wri tebunct tom) (int wena: Ine 


int returnValue; 
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bucketElement* fdPtr = £dTable. lookup (tid)-— 


fd_type thisFdType; 


//if ptr is nullg@then this™is stds. 
if (f£4aPtr) { 

thisFdType = fdPtr->type; 
i 


else{ 


thisFdType = TERMINAL_IO; 


}//end if 


//if this is awrite from across the network, check latency and b/w 


if(thisFdType == NETWORK_IPC) { 


//append flags to the front and rear of the message 


returnValue = networkWrite(fd, buf, len, writeFunction) ; 


resourceMonitor.updateNetWrites (returnValue) ; 
} 
else{ 

//pass the system call on 


returnvalue = 1 ( *wreiterunetren) (fa, Oee-) ten. 


if (thisFdType==LOCAL_FILE) { 
resourceMonitor.updateLocWrites (returnValue) ; 


} 
else if(thisFdType == TERMINAL_IO) { 


resourceMonitor.updateTerminalWrites(returnValue) ; 
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} 

else if{thisFdType == REMOTE_FILE) { 
resourceMonitor.updateDistWrites (returnValue) ; 

} 

eee //by elimination, must be local IPC 
resourceMonitor.updateLocIPCWrites (returnValue) ; 

}//end if 


}/fend if 


¥/eecuenis ene Sizerorteeniowwrite 
return (returnValue); 


}//end writeWrapper () 


// Function: closeWrapper () 
// Purpose: Calls the system’s close call, cleans up the 


Ty fd data structure. 


V7 


int closeWrapper(int fd, int(*closeFunction) (int) ) 


{ 


int returnValue = d{fellocerunct lone] 
#ifdef MSHN_ DEBUG 


cout<<"--inside closeWrapper--"<<endl; 


#tendif 


a5 


if (returnValue == 0) { 
fdTable.remove (fd); 


9) vesatel alae 


#ifdef MSHN_ DEBUG 
fdTable.printHashTable(); 
cout<<"--about to leave close wrapper--"<<endl<<endl; 


#endif 


return(returnValue) ; 


}//end closeWrapper () 


// Function: openWrapper () 

// Purpose: Calls the system’s open system call. Evals what 
rigs type of file is opened (if the open is successful), and 
i] stores this information in the fdTable hash table data 


7 SELUCEUKEe. 


LY 
int openWrapper(const char *file_name, int flag, int mode, 


ine (*openEunmetion) (Const char) Ime, nee 


int returnValue = (*openFunction) (file_name, flag, mode); 


#1ifdef MSHN_DEBUG 


96 


cout<<"--inside openWrapper: fd=> "<<returnValue<<"file=> "; 
cout<<file name<< " --"<<endl; 


Pemeit 


if (returnValue > 0) { 
fdTable.make_entry(returnValue, getFDType(returnValue) ) ; 


}//end if 


#ifdef MSHN_DEBUG 
fdTable.printHashTable(); 
cout<<"--about to leave open wrapper--"<<endl<<end1; 


#tendif 


return (returnValue) ; 


}//end openWrapper () 


// Function: socketWrapper () 

// Purpose: Calls the system’s socket sys call. Evaluates what 
je type of socket is opened (if the open is successful), and 
1g Stores this information in the fdTable hash table data 

ii Structune. 
ee 

kf 

int socketWrapper(int domain, int type, int protocol, 


int (*Seckerrunction) (ime. ant. 2ne) ) 


oF 


unsigned long args[3]; 


args[0] = domain; 

args[1] = type; 

args[{2] = protocol; 

int returnValue = (*socketFunction) (domain, type, protocol); 


fd_type fdType; 


#ifdef MSHN_DEBUG 


cout<<"--inside socketWrapper--"<<endl; 


cout<<"return value => "<<returnValue<<endl; 


#endif 


if (returnValue > QO) { 


//need to 


LP (domain 


fdType 
} 
else{ 
fdType 


V/ yen. as 


refine this to catch bind/access syscall to ip addr 


—— Z) { //2==PF_INET 


= NETWORK_IPC; 


= LOCAL_IPC; 


fdTable.make_entry(returnValue, fdType) ; 
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i /end 1£ 


#ifdef MSHN_DEBUG 
fdTable.printHashTable(); 
cout<<"--about to leave socketWrapper--"<<endl<<endl; 


#endif 


return (returnValue) ; 


}//end socketWrapper () 


// Function: acceptWrapper() 

// Purpose: Calls the system’s accept system call, Evaluates what 
age type of accept is opened (if the open is successful), and 

ii stores this information in the fdTable hash table data 

WE structure. 
ee 

a 

int acceptWrapper(int socket, struct sockaddr* addr, int* addrlen, 


int (*acceptFunction) (int, SERuctE sockaddr”. Ime») 


int returnValue; 


fd_type fdType; 
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bucketElement* fdTablePtr; 


//pass on the system call 
returnValue = (*acceptFunction) (socket, addr, addrlen); 
#ifdef MSHN_DEBUG 

cout<<"--inside acceptWrapper--"<<endl; 

cout<<"return value => "<<returnValue<<endl; 


#endift 


if (returnValue > 0) { 


unsigned long acceptAddress; 


intavsteoca l- 


//this is something of a hack; normally, would dereference the 
// struct sockaddr* addr, but we’d need to include socket.h, and 


// that would screw up our redefinition of the socket calls. 
ines 


// function gets the address of the machine which we accepted a 
// connection 
char* tempPtr = (char*)addr; 


//memcpy (&acceptAddress, (tempPtr + sizeof(short) + sizeof(u_short)), 
4); 


//isLocal = resourceMonitor.isHostLocal (acceptAddress) ; 


isLocal = 1; // ensures reads, writes are local 
//update the appropriate entries in the fd table 


if(isbLecal)< 


//create an entry for this fd in the fd hash table; include 
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ah; 


of Gitel Veta st= 
fdTable.make_entry(returnValue, LOCAL_IPC) ; 
} 
else{ 
J/j/ereate an entry for this fad im the fd hash peeies enter 
// fd type, ip address of remote machine, clockOffset between 
// the remote machine and this machine. 
fdTablePtr = fdTable.make_entry(returnValue, NETWORK_IPC) ; 


fdTablePtr->ipAddress = acceptAddress; 


fdTablePtr->clockOffset = getClockOffset (acceptAddress, PORT, 


}//end if 


}//end if 


#ifdef MSHN_DEBUG 


fdTable.printHashTable(); 


cout<<"--about to leave acceptWrapper--"<<endl<<endl; 


#endif 


return (returnValue) ; 


}//end acceptWrapper () 


TT  _  __ _____ 


// Function: connectWrapper (int, struct sockaddr*, int*) 


// Purpose: Calls the system’s connect system call, Evaluates what 
i/ type of connect is opened (if the open is successful), and 
17, stores this information in the fdTable hash table data 
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ees 


int connectWrapper(int socket, 


Structure: 


struct sockaddr* addr, int addrlen, 


int (*connect Function) (int, stisuceesockaddne amt 


int returnValue; 


fd_type fdType; 


bucketElement* fdTablePtr; 


#ifdef MSHN_DEBUG 


cout<<"--inside connect wrapper--"<<endl; 


#tendif 


//pass on the system call 


returnValue = (*connectFunction) (socket, addr, 


if (returmvalue =— 0) { 


long connectAddress; 


fat lsokhocal, 


addrlen); 


//this is something of a hack; normally, would dereference the 


// struct sockaddr* addr, but we’d need to include socket.h, and 


// that would screw up our redefinition of the socket calls. 


thas 


// function gets the address of the machine which we connected a 


Ve eGeonnect ion 


char* tempPtr = (char*)addr; 

// memcpy (&connectAddress, (tempPtr + sizeof(short) + 
sizeof(u_short)), 4); 

yy wehocale= TesoureceMoni tor. 1SHOstlLocal (connectAddress): 

Tslecal — 1) )//ensures treads writes arewmliae um 


if (isLocal) { 
//create an entry for this fd in the fd hash table; include 
j// EAS ype 
fdTable.make_entry(socket, LOCAL_IPC); 
} 
else{ 
//create an entry for this fd in the fd hash table; enter 
// €d type, ip address of remote machine, clockOffset between 
// the remote machine and this machine. 
fdTablePtr = fdTable.make_entry(socket, NETWORK_IPC); 
fdTablePtr->ipAddress = connectAddress; 


getClockOffset (connectAddress, PORT, 


FaTablePter—-clockOrrset 
ie 


}//end if 


}//end if 


#ifdef MSHN_DEBUG 
fdTable.printHashTable(); 


cout<<"--about to leave 


connect wrapper--"<<endl<<endl; 


#endift 


return(returnValue) ; 
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}//end connectWrapper () 


7 
#ifdef LINUX 


// PuRnRcELOn weer ead) 
// Purpose: "catches" the read system call, calls readWrapper () 


int. = read(ime fd, char* burt, int len) 


#ifdef MSHN_DEBUG 
cout<<"CAUGHT A __read()"<<endl; 
#tendif 
return(readWrapper (fd, buf, len, _ READ)); 


}//end __read() 


Yn ee 

// Fiumetion. = write(inte fd, char bus simeelen) 

// Purpose: "catches" the write system call, calls writeWrapper(). 
ff mr rr 

Ti 


Int: Wrree line ta, Char* but, tet den) 


{ 
return(writeWrapper(fd, buf, len, __WRITE)); 


}//end __write() 


| | mr 
// Function: _close(int fd) 
// Purpose: "catches" the _ close system call. Calls 


Ue closeWrapper(). 


int ©<-close(int fa){ 
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#ifdef MSHN_DEBUG 

Cout<- CAUGHT A __ close() "<<endl1; 
#endif 
return (closeWrapper(fd, CLOSE) ); 


iV vena  cilose 
#endif 


Ta 


JP Funerion:. read(int fd, enar* bur, int Jen) 
// Purpose: "catches" the _read system call, calls the 
ve readWrapper (). 
ime ._read(ant £4, 7char*® butpeint len 

#ifdef MSHN_DEBUG 

cout<<"CAUGHT A _read()"<<endl; 
#endif 
return(readWrapper(fd, buf, len, _—READ)); 


i / Chemeneacigts ECewchan* Ibu be mame Len 


// Function: cwrite(int fd, char “but, eine len) 
// Purpose: "catches" the _write system call, calls the 


// writeWrapper(). 


hal 


int 2writel(ine Ed, chars = butneamt len) 
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return(writeWrapper(fd, buf, len, _WRITE)); 


}/ fend write tint td). char*® but, ameter 


(PES SS 6 SS SSS SS 9 9S SSS SS = SSS SSS SSS SSS = SSS SS SS SSS SS SSS a= 
// Function: _open(const char *file_name, int flag, int mode) 
// Purpose: "catches" the _open system call, calls the 


i openWrapper(). 


Lf 
int _open(const char *file_name, int flag, int mode) 
{ 
#i1fdef MSHN_DEBUG 
cout<<"CAUGHT A _open({) "<<endl; 
#endif 


return (openWrapper (file_name, flag, mode, _OPEN) ); 


}//end _open 


[fsa St a SSS SSS so Se ee eee 
// Function: _close(int fd) 
// Purpose: "catches" the _close system call. Calls the 


77 closeWrapper(). 
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imeeeelose(int f£d) 
{ 
#ifdef MSHN_DEBUG 
cCout<<"GAUGHT A _close()@<<endl; 
Hendit 
return (closeWrapper(fd, _CLOSE)); 


}/f/end _close 


(nan a = SSS SSS + SSS SS 
j// Fumetion: socket (intwdomain ment Eype, int protocel) 
// Purpose: "catches" the _socket system call, calls the 


ad socketWrapper(). 


ig 
int _socket(int domain, int type, int protocol) 
{ 
#ifdef MSHN_ DEBUG 
cout<<"CAUGHT A _socket()"<<endl; 
#endif 


return(socketWrapper (domain, type, protocol, _SOCKET) ); 


}//end _socket(int domain, int type, int protocol) 


// Function: _accept(int socket, struct sockaddr* addr, int* addrlen) 


// Purpose: "catches" the _accept system call, calls the 
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Te acceptWrapper(). 


if 
int _accept(int socket, struct sockaddr* addr, int* addrlen) 
{ 
#ifdef MSHN_DEBUG 
cout<<"CAUGHT A _accept()"<<endl; 
#endif 


return(acceptWrapper (socket, addr, addrlen, ACCEPT) ); 


}//end _accept(int socket, struct sockaddr* addr, int* addrlen) 


TE TN NS NN NN a a a I SSS SS 
// Funetion: __accept(int socket, struct sockaddr* addr, int* addrlen) 
// Purpose: "catches" the __accept system call, calls the 


// acceptWrapper(). 


ae 


int __accept(int socket, struct sockaddr* addr, int* addrlen) 


{ 
#ifdef MSHN_DEBUG 
cout<<"CAUGHT A __accept()"<<endl; 
#endif 


return (acceptWrapper (socket, addr, addrlen, _ ACCEPT) ); 


}//end __ accept(int socket, struct sockaddr* addr, int* addrlen) 
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/f Punetion. — connect (int socket, struct sockaddr* addr, int* addzlen) 


// Purpose: "catches" the _connect system call, calls the 
ys comaemihianweae 0) - 

|| = - =e a a Oe ee eS ae eee ao eee 
(iE 


int _connect(int socket, struct sockaddr* addr, int addrlen) 
{ 
#ifdef MSHN_DEBUG 
cout<<"CAUGHT A _connect() "<<endl; 
#endif 


return(connectWrapper (socket, addr, addrlen, _CONNECT) ); 


}//end _connect(int socket, struct sockaddr* addr, int* addrlen) 


// Function: _connect2(int socket, struct sockaddr* addr, int* addrlen) 


// Purpose: "catches" the _connect2 system call, calls the 
ie | connectWrapper(). 
ee 
a 


int .connect2 (int socket, struct sockaddr* addr, int addrilen) 
{ 
#ifdef MSHN_DEBUG 
cout<<"CAUGHT A _connect2()"<<endl; 
#endif 


return (connectWrapper(socket, addr, addrlen, _CONNECT2) ); 
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}//end _connect2(int socket, struct sockaddr* addr, int* addrlen) 


#endif 


// BUNCtion: read (int fa, ‘char: burt, 2m, Ven) 
// Purpose: "catches" the read system call, calls the 
so), readWrapper(). 


iy 
inet reaaiint sta. cuan but tat len) 
{ 

#ifdef MSHN_ DEBUG 

cout<<"CAUGHT A read()"<<endl; 

#tendif 

return(readWrapper(fd, buf, len, READ)); 
}//end read(int fd, char* buf, int len 


[faa ee eG a 
// Function: write{int fa? char* “but, ane Jen) 
// Purpose: "catches" the write system call, calls the 


Vs writeWrapper(). 


es 
int woreeiint. [d, chae* sour, inc even) 


{ 
return(writeWrapper(fd, buf, len, WRITE)); 


}/fend write (int. fd, char soul. 145 -len 


ff === === == << ~~ = = ae 
// Function: open(const char *file_name, int flag, int mode) 
// Purpose: "catches" the open system call, calls the 


// openWrapper(). 
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es 
int open(const char *file_name, int flag, int mode) 


{ 
#ifdef MSHN_DEBUG 
cout<<"CAUGHT A open() "<<endl; 
Rendif 


return (openWrapper(file_name, flag, mode, OPEN)); 


}//end open 


, fe — — — SS 
// Functienm close(int £d) 
// Purpose: "catches" the close system call, calls the 


Lh closeWrapper(). 
fa 
dmteelose (int. tap 
#ifdef MSHN_DEBUG 
cout<<"CAUGHT A close()"<<endl; 
#endif 
return(closeWrapper(fd, CLOSE) ); 


}//end close 


[OS SS SS SS SS SS SS SS SS SSS SS SS SS SS SS 
// Sunction: secket (int domain, int vtyvpe,, 2meaprortocol) 
// Purpose: "catches" the socket system call, calls the 


ff socketWrapper(). 
// 
int socket(int domain, int type, int protocol) 
#ifdef MSHN_DEBUG 
cout<<"CAUGHT A socket()"<<endl; 
#endif 


return (socketWrapper (domain, type, protocol, SOCKET)); 


}//end sockét(int domain, intvtyoe;, int pretecol) 


MW 


// Function: accept(int socket struet sockaddr* addr, int* addrlen) 
// Purpose: "catches" the accept system call, calls the 


// acceptWrapper(). 
int accept(int socket, struct sockaddr* addr, int* addrlen) 
{ 
#ifdef MSHN_DEBUG 
cout<<"CAUGHT A accept()"<<endl; 


#endif 


return(acceptWrapper (socket, addr, addrlen, ACCEPT)); 


}//end accept(int socket, struct sockaddr* addr, int* addrlen) 


f fmm nm nn a ee = == - = 

// Function: connect(int socket, struct sockaddr* addr, int* addrlen) 
// Purpose: "catches" the connect system call, calls the 

eve connectWrapper 

| fmm mm nr ener 

// 


int connect (int socket, struct sockaddr* addr, int addrlen) 
{ 
#ifdef MSHN_ DEBUG 
cout<<"CAUGHT A connect()"<<endl; 


#Hendif 


return(connectWrapper (socket, addr, addrlen, CONNECT) ); 


}/J/end connect (int socket, struct sockaddr* addr, 2me* addrlen) 


lf === = == == = = = ee 
//PFUNnCEION: Vold. exit (int Sseacus) 
// Purpose: "catches" the exit system call, calls the 


ves exitWrapper(). 


Mos 
void _exit(int status) 
{ 
#ifdef MSHN_DEBUG 
printf ("--inside _exit wrapper--\n"); 
#endif 
//pass the system call on 
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exitWrapper (status, _EXIT); 
J Wi mOt- return 
Pe ndmeCkiiti aime: Status) 


Oy Se Se SS SSS SS SSS SS SS SS 0 SS SS SSS 
/7 “Eumction: void exit () 

// Purpose: "catches" the exit system call, calls the. 

es exitWrapper() function. 

a ee ee we eee 
Vy 


VOld Cxitelaint status) 


{ 
#ifdef MSHN_DEBUG 
printf("--inside exit function--\n"); 
#endif 
exitWrapper (status, EXIT); 


}/f/end exit(int status) 


//end file MSHN_syscall_lib.cc 


bs 
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APPENDIX D. MSHN_MONITOR_RRD_CLASS.CC 


Code for sending results of MSHN Client Library Wrapper to an files. 
of alee i eee ene XXX x RK RE KAR AKER 


// File: MSHN_monitor_RRD_Class.cc 

// Name: Matt Schnaidt (modified with permission by W Porter) 
// Operating Environment: Linux™Z7e" 297" nx 4.1.4 

// Compiler: g++ for Linux, Sun OS 4.1.4 

// Date: 25 Feb 99 


(lf 
// Description: Records resource usage information, and 
ae updates the resource server when called. 


// Inputs: Number of bytes read or written. 
// Assumptions: Calling progams input proper data type. 


// Warnings: none. 

ty Re RR a EAE KA 
Va 

#include <stdio.h> //for FILE 


#include <iostream.h> 
#include <fstream.h> 
finelude <stdlib.h> 
finelude <string. i= 
#include <sys/time.h> 


#include <sys/resource.h> //for rusage 
#include <netdb.h> //for get hostent 
Finclude <unistd.h> //for gethostname 
#include <netinet/in.h> jjmene loicexo ll)! 


#include "MSHN_monitor_RRD_Class.h" 
#include "MSHN_types.h" 
#include "MSHN_utility.h" 


//local prototype 
char* getPsData(int, char*); 


#ifdef SUN55 


extern eC raf 

extern int getrusage(int, struct rusage*); 
ti (ena -externaue. 
#endif 


#ifdef SUN4 
extern "C" { 
extern int getrusage(int, struct rusage*); 
iif fends extern. '"C. 
#endif 


// Function: MSHN_monitor_RRD_Class::MSHN_monitor_RRD Class(): 
// Return Value: none. 


// Parameter: none. 
// Purpose: User-defined default constructor; initializes 
ese resource data members. 


5) 


[7 
MSHN_monitor_RRD_ Class: :MSHN_monitor_RRD_Class(): 
$izeLocReads (0), numLocReads (0), 
sizeLocWrites(0), numLocWrites(0), sizeDistReads(0), 
numDistReads (0), 
sizeDistWrites(0), numDistWrites(0), timeDistReads(0.0), 
numTerminalReads(0), sizeTerminalReads(0), timeTerminalReads(0.0), 
numTerminalWrites(0), sizeTerminalWrites(0), numNetReads(0), 
sizeNetReads(0), sizeNetWrites(0), numNetWrites(0), 
timeNetReads (0.0), 
thisAddress (0) 


const int HOST_NAME SIZE = 100; 

char hostName [HOST_NAME_SIZE]; 

gethostname (hostName, HOST_NAME_SIZE) ; 

Struct hostent* thisHostPtr = gethostbyname(hostName) ; 

memcpy (&thisAddress, thisHostPtr->h_addr, thisHostPtr->h_length) ; 
}//end MSHN_monitor_RRD_Class: :MSHN_monitor_RRD_Class() 


J// CESEEUCEOM 
MSHN_monitor_RRD_Class: :~MSHN_monitor_RRD_ Class () { 


}//end MSHN_monitor_RRD_Class: :~MSHN_monitor_RRD_Class() 


| f mr er 

// Function: MSHN_monitor_RRD_Class::isHostLocal(const int& HOST) 
// Return Value: int, 1 if host is local, 0 otherwise. 

// Parameter: unsigned long, HOST, the host address to compare 
Lt to this one. MUST be in network byte ordering. 

// Purpose: If HOST is the same address as this machine, 

Vey) returns 1; else returns 0. 

TL a eee 

hy 


int MSHN_monitor_RRD_Class::isHostLocal (const unsigned long& HOST) 
{ 


return(thisAddress == HOST); 


}//end int MSHN_monitor_RRD_Class::isHostLocal(const int& HOST) 


// Function: MSHN_ monitor _RRD Class:: updateLocReads(int& len) 

// Return Value: none. 

// Parameter: int, the size of the read. 

// Purpose: Updates total bytes read locally and total number of 
1s reads. 


He 
void MSHN_monitor_RRD_Class: :updateLocReads (int& ern) 


{ 


//only increase if read is successful 
if (len>0) { 
SizeLocReads += len; 
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}//end if 


J//count number of reads -- successful or not 
numLocReads++; 


#ifdef MSHN_DEBUG 


cout<<endl<<"--inside updateLocRead--"<<endl; 
cout<<"Total size of local reads = " << sizeLocReads << endl; 
cout<<"--leaving updateLocRead--"<<endl<<endl; 

#endif 

Gebunn; 


}//end MSHN_monitor_RRD_Class: :updateLocReads (int&) 


// Function: MSHN_monitor_RRD_Class: :updateLocWrites(int& len) 

// Return Value: none. 

// Parameter: int, the size of the write. 

// Purpose: Updates total bytes written locally and total number of 
ie writes. 

TE aa a a ll a oo ca a al mc ela dpe alo a a oe 

Wy 

void MSHN_monitor_RRD Class: :updateLocWrites(int& len) 

{ 


//only increase if write is successful 


Tetlen>0) £ 
SizeLocWrites += len; 
}//end if 
//count number of reads -- successful or not 
numLocWrites++; 


#ifdef MSHN_DEBUG 
cout<<endl<<"--inside updateLocWrites--"<<endl; 
cout<<"Size of local writes = " << sizeLocWrites << endl; 
cout<<"--leaving updateLocWrites--"<<endl<<endl; 

#endif 


}//end MSHN_monitor_RRD_ Class: :updateLocWrites (int&) 


// Function: MSHN_monitor_RRD_Class:: updateDistReads(int& len, double& 
time) 

// Return Value: none. 

// Parameter: int, the size of the read. 

// Purpose: Updates total bytes read locally and total number of 

Ti} reads on network file server. 


es 
void MSHN_monitor_RRD_ Class: :updateDistReads(int& len, double& time) 


{ 


//only increase if read is successful 
if (len>0O) { 
sizeDistReads += len; 


Gs 


ElmeDistReads += time; 
}//end if 


//count number of reads -- successful or not 
numDistReads++; 


#ifdef MSHN_DEBUG 


_cout<<endl<<"--inside updateDistRead--"<<endl; 
cout<<"Total size of network fs reads = " << sizeDistReads << 
endl; 
cout<<"--leaving updateDistRead--"<<endl<<endl; 
#endif 
BeEWUrEn; 


}//end MSHN_monitor_RRD_ Class: :updateDistReads(int&, double& time) 


// Function: MSHN_monitor_RRD_ Class: :updateDistWrites(int& len) 
// Return Value: none. 


// Parameter: aint, the size of the write. 

// Purpose: Updates total bytes written and total number of 
hon writes on network file server. 

| A aaieneaieteaaietetententedenteateianlantentetetientetententntedianmten aetna nedmnaienananananmnianantantedameiananaenteten enter 
// 


void MSHN_monitor_RRD_ Class: :updateDistWrites(int& len) 
{ 
//only increase if write is successful 
if (len>0) { 
sizeDistWrites += len; 
}//end if 


//count number of reads -- successful or not 
numDistWrites++; 


#ifdef MSHN_DEBUG 


cout<<endl<<"--inside updateDistWrites--"<<endl; 
cout<<"Size of network fs writes = " << sizeDistWrites << endl; 
cout<<"--leaving updateDistWrites--"<<endl<<endl; 

#endif 


}//end MSHN_monitor_RRD_ Class: :updateDistWrites (int&) 


// Function: MSHN_monitor_RRD_Class:: updateNetReads(int& len, double& 
time) 
// Return Value: none. 


// Parameter: int, the size of the read. 

// Purpose: Updates total bytes read locally and total number of 
LT reads on network file server. 

| Kariaieteirinteietntetetettetntentetententettetntettetetentenenmteteteteatetentatetetetetentnineteeneeeeenen 

// 


void MSHN_monitor_RRD_Class: :updateNetReads(int& len, double& time) 
{ 


//only increase if read is successful 
1f (len>0) { 
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SsizeNetReads += len; 
timeNetReads += time; 
}//end if 


//count number of reads -- successful or not 
numNetReads++; 


#ifdef MSHN_DEBUG 


cout<<endl<<"--inside updateNetRead--"<<endl; 
cout<<"Total size of network reads = " << sizeNetReads << endl; 
cout<<"--leaving updateNetRead--"<<endl<<endl; 
#endif 
metuuin: 


}//end MSHN_monitor_RRD_Class: :updateNetReads(int&, double& time) 


// Function: MSHN_monitor_RRD_Class: :updateNetWrites(int& len) 
// Return Value: none. 


// Parameter: int, the size of the write. 

// Purpose: Updates total bytes written and total number of 
vi, writes on network file server. 
SS SS 
es 


void MSHN_monitor_RRD_ Class: :updateNetWrites(int& len) 
{ 


/fonly increase if write is successful 
i1f(len>0) { 

sizeNetWrites += len; 
}//end if 


//count number of reads -- successful or not 
numNetWrites++; 


#ifdef MSHN_DEBUG 
cout<<endl<<"--inside updateNetWrites--"<<endl; 


cout<<"Size of network fs writes = " << sizeNetWrites << endl; 
cout<<"--leaving updateNetWrites--"<<endl<<endl; 
#endif 


}//end MSHN_monitor_RRD_Class: :updateNetWrites (int&) 


// Function: MSHN_monitor_RRD_Class:: updateLocIPCReads (int& len) 
// Return Value: none. 


// Parameter: int, the size of the read. 

// Purpose: Updates total bytes read locally and total number of 
// reads on network file server. 
ee - 

jf 


void MSHN_monitor_RRD_Class: :updateLocIPCReads (int& len) 
{ 


/fonly increase if read is successful 
if (len>0) { 
sizeLocIPCReads += len; 
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}7 fend 25 


//count number of reads -- successful or not 
numLocIPCReads++; 


#ifdef MSHN_DEBUG 


cout<<endl<<"--inside updateLocIPCRead--"<<endl; 
.cout<<"Total size of network reads = " << sizeLocIPCReads << 
endl; 
cout<<"--leaving updateLocIPCRead--"<<endl<<end1; 
#endif 
return; 


}//end MSHN_monitor_RRD_Class: :updateLocI PCReads (int&) 


// Function: MSHN_monitor_RRD_Class: :updateLocIPCWrites(int& len) 
// Return Value: none. 


// Parameter: int, the size of the write. 

// Purpose: Updates total bytes written and total number of 
Ed, writes on network file server. 

[ [aaa Sea SR SSS SSS tS a SS Se Se a a 
// 


void MSHN_monitor_RRD_Class: :updateLocIPCWrites(int& len) 
{ 


//only increase if write is successful 
if(len>0O) { 
SizeLocIPCWrites += len; 


hy) Gnas ale 

//count number of reads -- successful or not 

numLocIPCWrites+t+; 

nese (UO) 
cout<<endl<<"--inside updateLocIPCWrites--"<<endl; 
cout<<"Size of network fs writes = " << SsizeloclPCWrites ——senen 
cout<<"--leaving updateLocIPCWrites--"<<endl<<endl; 

joy), 7 (sq\6| Male 


}//end MSHN_monitor_RRD_Class: :updateLocIPCWrites (int&) 


// Function: MSHN_monitor_RRD_Class:: updateTerminalReads(int& len, 
double& time) 


// Return Value: none. 

// Parameter: aint, the size of the read. 

// Purpose: Updates total bytes read and total number of 

14, reads from terminal and time to do reads. 

| 7 Koteriertententententententententententententetentetententententententententententetentetetetentententententetetetetetetetententetetentetentetetententanten 
ag) 


void MSHN_monitor_RRD_Class: :updateTerminalReads(int& len, double& 
time) 
{ 

//only increase if read is successful 

1f (len>0) { 


120 


sizeTerminalReads += len; 
timeTerminalReads += time; 
fend ia 


//count number of reads -- successful or not 
numTerminalReadst+; 


#ifdef MSHN_DEBUG 


cout<<endl<<"--inside updateTerminalRead--"<<end1; 
cout<<"Total size of terminal reads = " << sizeTerminalReads << 
endl; 
cout<<"--leaving updateTerminalRead--"<<endl<<endl; 
#endif 
return; 


}//end MSHN_monitor_RRD_Class: :updateTerminalReads(int&, double& time) 


// Function: MSHN_monitor_RRD_Class:: updateTerminalWrites(int& len) 
// Return Value: none. 


// Parameter: int, the size of the write. 

// Purpose: Updates total bytes write and total number of 
hy writes from terminal and time to do writes. 

ae ee 8 ee 
// 


void MSHN_monitor_RRD_ Class: :updateTerminalWrites(int& len) 
{ 
//only increase if write is successful 
1f£ (len>O) { 
sizeTerminalWrites += len; 
Wi/end 2.t 


//fcount number of writes -- successful or not 
numTerminalWrites++; 


ue (0).t 
cout<<endl<<"--inside updateTerminalWrite--"<<endl; 
cout<<"Total size of terminal writes = " << sizeTerminalWrites << 
endi.- 
cout<<"--leaving updateTerminalWrite--"<<endl<<endl; 
}//end if 
return; 


}//end MSHN_monitor_RRD_Class: :updateTerminalWrites(inté&) 


// Function: MSHN_monitor_RRD_ Class: :updateResourceServer(int status) 
// Return Value: none. 
// Parameter: none. 


// Purpose: Updates the resource server with the total 

// resources used by this application. Updates by 

J7 application name. Currently, this is simply printed 

ae to the screen. Another student is developing a CORBA 
a! persistent object to act as the resource server. 

ff mmr en --- 


fey 
void MSHN_monitor_RRD_Class: :updateResourceServer(int status) 


{ 
int <checkPrd-; //this process’ Pid 


//creates output file 
ofstream OuweEputlLoc ("MSHN_ outta les.txe. 3) 20s: Joma 


//if debug is set, output debug info 
#ifdef MSHN_DEBUG 

cout<<endl<<"--inside updateResourceServer--"<<endl; 
#endif 


#ifndef OUTPUT_TO_FILE 
ostream& outputLoc = cout; 
#endif 


appEndTime = getClockTime() ; 


outputLoc<<"Simulating update to Resource Requirements 
Database"<<endl; 

outputLoc<<"Application name: "<<exeName<<endl; 

outputLoc<<"Input args: "<<inputArgs<<endl; 


iE(Csceatucs =— Om 
outputLoc<<"Application terminated normally"<<endl; 
} 
else { 
outputLoc<<"Application terminated abnormally, exit status = 
"<<status<<endl; 


}//end if 

outputLoc<<"-------- LOCAL FILE DATA --------- "<<end 1, 
outputLoc<<"Total Bytes Read: "<<silzeLocReads; 
outputLoc<<" Written: "<<sizeLocWrites<<endl; 
outputLoc<<"Number of Reads: "<<numLocReads; 
outputLoc<<" Writes : "<<numLocWrites<<endl; 
outputLoc<<"-------- NETWORK FILE DATA --------- "<<end1 ; 
outputLoc<<"Total Bytes Read: "<<sizeDistReads; 
outputLoc<<" Written: ”" <<sizeDistWrites<<end1l; 
outputLoc<<"Number of Reads: "<<numDistReads; 
outputLoc<<" Writes : " <<numDistWrites<<endl; 
outputLoc<<"Total Time Reading: "<<timeDistReads<<endl; 
outputLoc<<"-------- TERMINAL I/O DATA --------- "<<endl; 
outputLoc<<"Total Bytes Read: "<<silzeTerminalReads; 
outputLoc<<" Written: " <<sizeTerminalWrites<<endl; 
outputLoc<<"Number of Reads: "<<numTerminalReads; 
outputLoc<<" Writes : " <<numTerminalWrites<<endl; 
outputLoc<<"Total Time Reading: "<<timeTerminalReads<<endl; 
outputLoc<<"-------- NETWORK DATA £--------- "<<endl ; 
outputLoc<<"Total Bytes Read: "<<s1zeNetReads; 
outputLoc<<" Written: ” <<sizeNetWrites<<endl; 
outputLoc<<"Number of Reads: "<<numNetReads; 
SutpucLoec<<"' Writes = <<numNetWrites<<end1l; 
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output Loc<<"----=---- LOCAL IPC DATA  -<<<<=-==- ee eT G li: 


outputLoc<<"Total Bytes Read: "<<sizeLocIPCReads; 
OuLpULLOoc<<" Written: .” <<sizeLocIPCWrites<<endl; 
outputLoc<<"Number of Reads: "<<numLocIPCReads; 
outputLoc<<" Writes: " <<numLocIPCWrites<<end1; 


outputLoc<<endl<<"Application’s Wall Clock Runtime: "; 
outputLoc<<calcTimeDiff(&appStartTime, &appEndTime) ; 
outputLoc<<" seconds"<<end1; 


//this determines and outputs how long the process ran 
struct rusage rusageStruct, 


*rusagePtr = &rusageStruct; 


getrusage(RUSAGE_SELF, rusagePtr) ; 


double userTime = ((double) (rusagePtr->ru_utime.tv_usec)/1000000.) + 
rusagePtr->ru_utime.tv_sec; 

double sysTime = ((double) (rusagePtr->ru_stime.tv_usec)/1000000.) + 
rusagePtr->ru_stime.tv_sec; 

outputLoc<<end1 ; 

outputLoc<<"system cpu time = "<<sysTime<<" seconds"<<end1; 

outputLoc<<"user cpu time = "<<userTime<<" seconds"<<endl; 


"<<TrUuSagePeEr—->ru_maxrss<<" 


outputLoc<<"max res set size 
pages "<<end1; 

outputLoc<<"unshared memory 
pages"<<endl; 


"<<rusagePtr->ru_idrss<<" 


il 


outputLoc<<"page faults = "<<rusagePtr->ru_majflt<<endl; 
outputLoc<<"---at program termination, the memory stats were: 
*<<endl; 


#ifdef SUN55 

int. thisPid = "qgetpid(); 

outputLoc<<"num of physical pages in memory 
"“OSZ" )=<endl> 

outputLoc<<"num of pages in virtual memory 
"wsz")<<endl; 

outputLoc<<"num of pages in resident set = "<<getPsData(thisPid, 
"rss")<<endl; 
#endif 


"<<getPsData(thisPid, 


"<<getPsData(thisPid, 


return; 
}//end MSHN_monitor_RRD_Class:: updateResourceServer () 


i ee 

// Function: MSHN_monitor_RRD_Class::getPsData 

fd (int thisPid, char* psCommand) 
// Return Value: char* 

// Parameter: thisPid, psCommand 

// Purpose: gets process status (ps) data 

fof a SS a a a 8 = SS = 

es 


char* getPsData(int thisPid, char* psCommand) 


{ 


char thisPidPtrli2): //declares 4 char array for thisPid 


Pa) 


sprintf (thisPidPtr, "$d\0" ,thaisPid) >" /places”™ thicrid fuse naNaeciecn, 


char commandString[100] = "ps -p "; //declares variable for ps 
commands 

char *buffer = (char*) malloc(100*sizgeof(char)); //declares string 
POR 


/fMoDutEfer output 


strcat(commandString, thisPidPtr); 
Streat(commanastring, “ ~=o “*)s 
strcat (commandString, psCommand) ; 


char tempfilename[100] = "mshnTempFile"; //declares string for file 
name 

strceat(tempfilename, thisPidPtr) ; 

Sercae (commandaSt rane 9= >. J 


strcat(commandString, tempfilename) ; 
system(commandString); //uses command string for system -ps call 


ifstream inFile(tempfilename) ; 
inFile>>buffer; 


char remove[{100] = "rm"; 
strcat(remove," "); 


strceat(remove, tempfilename) ; 


system(remove); //removes file created to hold buffer contents 
returns (Surfer); 


}//end getPsData() 


[fe Sa oe eee = eae 

// Function: MSHN_ monitor_RRD Class: :enterExeName 

hey (const char* thisName) 

// Return Value: none. 

// Parameter: thisName. 

// Purpose: Sets exeName equal to thisName 

La a a a a 
[q 


//assign the executable name to exeName 
void MSHN_monitor_RRD_ Class: :enterExeName(const char* thisName) 
i 
strepy(exeName, thisName) ; 
BeEUrN; 
}//end MSHN_monitor_RRD_Class::enterExeName(const char* thisName) 


ff == ee ee 
// Function: MSHN_monitor_RRD_Class: :getExeName 

as () 

// Return Value: none. 

// Parameter: thisName. 

// Purpose: gets thisName 

| [mm nr ee === - 


a 
char* MSHN_ monitor_RRD_ Class: :getExeName () 
{ 
return (exeName) ; 
}//end MSHN_monitor_RRD_Class: :getExeName () 


[ee 2 ee See eee 2 6 eS = = = 
// Function: MSHN_monitor_RRD_Class::enteriInputArgs 

ys (const char* theseArgs) 
// Return Value: none. 

// Parameter: char*, theseArgs a string containing all input 
yy arguments. 

// Purpose: Sets inputArgs equal to theseArgs 

Ce a a a 
ee 


void MSHN_monitor_RRD_Class::enterInputArgs(const char* theseArgs) 
{ 
strcepy(inputArgs, theseArgs); 
return; 
}//end MSHN_monitor_RRD_Class::enterInputArgs(const char* theseArgs) 


//end file MSHN _monitor_RRD Class.cc 


[fmm rrr 

// Function: MSHN_monitor_RRD Class: :enterAppStartTime 

lee (const struct timeval* startTime) 
// Return Value: mone. 


// Parameter: const struct timeval*, the start timeof the 
ee application. 
// Purpose: sets appStartTime equal to startTime. 
| fmm rr rr rrr ene ----- 
Se //enter the start time of this application 
void MSHN_monitor_RRD_Class: :enterAppStartTime(const struct timeval* 
startTime) 
{ 
appStartTime.tv_sec = startTime->tv_sec; 
appStartTime.tv_usec = startTime->tv_usec; 


}//end MSHN_monitor_RRD_Class::enterAppStartTime() 





APPENDIX E. RUN SCRIPT FOR DEMO300 


“sh” run script for Demo300 that launches EADSIM executables, sets clockserver 
for wrapper, and resets random number generator seed by tying it to the system clock. 


#!/bain/sh 


C3 IDATA=/users/work4/nwporter/eadsim/data 
export C3IDATA 


Scenario=Demo300 

1£f [ S$# -eq 1 ]; then 
Scenario=S$l 

ca 


CrashPath="/users/work4/nwporter/crash" 
ExePath="/users/work4/nwporter/eadsim/execute/SUN2" 


clockServer & 


cd SCrashPath/c31 
SExePath/c31 $Scenario 0 ‘date ‘'+%H%M%S'*‘ & 


sleep 1 


ca SCrashPath/det 
SExePath/detect S$Scenario 0 ‘date '+%H%M%S’°* & 


sleep 16 


cd $CrashPath/fp 
SExePath/fp $Scenario & 
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APPENDIX F. RUN SCRIPT FOR TRANSFER OF DATA 


“sh” run script to transfer wrapper and simulation results for each Monte Carlo 
run to a Separate directory. Script assigns unique “run number” extension to each file of 


output data associated with a given run. 


#!/bin/sh 

1f [ ! "S#" -eq 1 ] 

then 
echo "Usage: $0 version_number" 
exit 1 

fe 


RUN. VER="Si" 
ROOT_DIR=/users/work4/nwporter / 


cd $S{ROOT_DIR:?}/eadsim 


if [| -f mcreports/Threatsum.runS{RUN_VER:?} ] 

then 
echo "Version S{RUN_VER:?} of data files exist" 
exit 1 

ime 


cp Threatsum.engrpt mcreports/Threatsum.run$S{RUN_VER: ?} 

cp BlueAction.engrpt mcreports/BlueAction.runsS {RUN_VER: ?} 
cp RedAction.engrpt mcreports/RedAction.runs {RUN_VER: ?} 

cd /users/work4/nwporter/crash/ 

cd ie3si 

cp MSHN_outfile.txt 
/users/work4/nwporter/eadsim/mcreports/c3i_outfile.run$ {RUN 
LVER= 2} 

cd /users/work4/nwporter/crash/ 

CG) Ep 

cp MSHN_outfile.txt 
/users/work4/nwporter/eadsim/mcreports/fp_outfile.runs {RUN_ 
VER: ? } 

cd /users/work4/nwporter/crash/ 

cd det 

cp MSHN_outfile.txt 
/users/work4/nwporter/eadsim/mcreports/det_outfile.run${RUN 
LVEBR se?) 

cd /users/work4/nwporter/eadsim/run 

Gares1 
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cp Demo300_1.c3ipstat 
/users/work4/nwporter/eadsim/mcreports/c3ipstat.runs {RUN_VE 
Ree} 

cd /users/work4/nwporter/eadsim/run 

ca detect 

cp Demo300_1.detpstat 
/users/work4/nwporter/eadsim/mcreports/detpstat.run$ {RUN_VE 
R:?} 

cd /users/work4/nwporter/eadsim/run 

eG, £p 

cp Demo300_1.fppstat 
/users/work4/nwporter/eadsim/mcreports/fppstat.runs {RUN_VER 
ee, 

cp Demo300_1.stathdr 
/users/work4/nwporter/eadsim/mcreports/stathdr. runs {RUN_VER 
ey, 
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APPENDIX G. RUN SCRIPT FOR TRANSFER OF DETERMINISTIC DATA 


“sh” run script to transfer wrapper and simulation results for each deterministic 
run to a separate directory. Script assigns unique “run number” extension to each file of 


output data associated with a given run. 


#!/bin/sh 

if [ ! "S#" -eq il ] 

then 
echo "Usage: $0 version_number" 
exit 1 

fa 


RUN_VER="Si" 
ROOT_DIR=/users/work4/nwporter/ 


cd S{ROOT_DIR:?}/eadsim 

if [ -f£f dreports/c3ipstat.runsS{RUN_VER:?} ] 

then 
echo "Version S{RUN_VER:?} of data files exist" 
exit 1 

igi 


cd /users/work4/nwporter/eadsim/run 

Gch G33 

cp Demo300_1.c3ipstat 
/users/work4/nwporter/eadsim/reports/c3ipstat.run$ {RUN_VER: 
oF 

cd /users/work4/nwporter/eadsim/run 

cd detect 

cp Demo300_1.detpstat 
/users/work4/nwporter/eadsim/dreports/detpstat.runs${RUN_VER 
22 } 

cd /users/work4/nwporter/eadsim/run 

ea ie 

cp Demo300_1.fppstat 
/users/work4/nwporter/eadsim/dreports/fppstat.runs {RUN_VER: 
ca: 
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APPENDIX H. MSHN WRAPPER OUTPUT: MONTE CARLO TRIALS 


Note: Only data that generated a distribution is displayed. 





Descriptive Statistics 

Variable: C3l User 

CPU Time 

Anderson-Darling Normality Test 
A-Squared: 0.183 
P-Value: 0.886 
Mean I APA Wed 
StDev 0.8383 
Vaniance 0.702667 
Skewness 0.176778 
Kurtosis 5.54E-03 
N 30 
Mnimum 15.8600 
1st Quartle 17.2450 
Nedan 17.6650 
3rd Quartile 18.2850 
NVexmum 19.7400 


95% Confidence Interval for Mu 


~ 44 


5 





‘ 


; ] l i i 
i75 76 77 78 i79 180 181 
| 





05° Confidence Interval for Medan 


Descriptive Statistics 


17.4047 


0.6676 


17.3637 


95% Confidence Interval for Mu 


18.0307 


95% Confidence Interval for Sgma 


1.1269 


95% Confidence Interval for Median 


18.0871 


Variable: FP User 








CPU Time 
Anderson-Darling Normality Test 
A-Squared 0.545 
P-Value 0.148 
Mean 17.1253 
StDev 0.8911 
Variance 0.794012 
Skewness 0.991135 
Kurtosis 2.26742 
N 30 
Mnimum 15.3600 
1st Quartile 16.6200 
Vedian 16.8950 
ard Quartile 17.6450 
95% Confidence Interna! for Mu Vexmum 20.1300 
or eee ea ete 
; " : 16.7926 17.4581 
forte, Waoe 16c2 V7 Ce wi 722° Vee “4. 17.22 re : 
) : ; ; , : : - 95% Confidence Interval for Sigma 
0.7097 1.1979 
: 
95% Coniidence Imenai for Vedian 95% Comidence Interval for Median 
16.7869 17.4603 
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Descriptive Statistics 
Variable: Detect User 





| | 
15.85 159 160 1615 1625 1625 1645 1655 16.6 
| 








Fg 
= 
* 


re . 


95% Confidence Interval for Vadian 





Descriptive Statistics 


16.0199 


0.6316 


15.8823 


CPU Time 

Anderson-Darling Normality Test 
A Squared: 0.506 
P-Value: 0.186 
Mean 16.3160 
StDev 0.7930 
Vanance 0.628887 
Skewness 0.893720 
Kurtosis 0.675003 
N 30 
Mnimum 14.9700 
1st Quartile 15.7625 
Median 16.2150 
3rd Quartile 16.7700 
Vexmum 18.6100 


95% Confidence Interval for Mu 


16.6121 


95% Confidence Interval for Sigma 


1.0661 


95% Confidence Interval for Median 


16.5026 


Variable: C3l System 





95% Confidence Internal for Mu 





95% Confidence Interval for Median 
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2.97021 


0.11899 


2.96000 


CPU Time 
Anderson-Darling Normality Test 
ASquared 0.357 
P-Value: 0.433 
Mean 3.02600 
StDev 0.14940 
Variance 2.23E-02 
Skewness -2.0E-01 
Kurtosis -3.3E-01 
N 30 
Minimum 2.67000 
1st Quartile 2.92250 
Vedian 3.05000 
3rd Quartile 3.12250 
Vaxmum 3.33000 


95% Confidence Interval for Mu 


3.08179 


95% Confidence Interval for Sigma 


0.20085 


95% Confidence Interval for Median 


3.10771 


Descriptive Statistics 


Variable: FP System 








CPU Time 
Anderson-Darling Normality Test 
A-Squared: 0.513 
| P-Value: 0.179 
} Mean 3.19667 
SwDev 021979 
Vanance 483E02 
Skewness -5.7E-01 
Kurtosis 0.106161 
N 30 
Mnimum 2.58000 
1st Quartile 3.02000 
NVedian 3.23500 
3rd Quartile 3.35750 
95% Confidence Interval for Mu Vexmum 3.58000 
ee Cee 
| 3.11460 3.27874 
— ete a oe 95% Confidence Interval for Sigma 
al ane 
: ~ 95°% Confidence Interval for Median 
5% Conti 
Confidence Interval for Median 3.05229 3.33543 


Descriptive Statistics 


Variable: Detect System 











CPU Time 
Anderson-Danling Normality Test 
A-Squared: 0.528 
P-Value: 0.164 
Mean 5.85567 
StDev 0.43027 
Variance 0.185129 
Skewness 0.833605 
Kurtosis 0.858158 
N 30 
Mnimum 5.10000 
1st Quartile 5.53500 
NVedian 5.73000 
3rd Quartile 6.10250 
95% Confidence Interval for Mu vee Tees 
ee 95% Confidence Interval for Mu 
5.69500 6.01633 
95% Confidence Interval for Sigma 
0.34267 0.57841 
95% Confidence Interval for Median 
95% Confidence Interval for Medi 
ni:dence Interval jor Medan 5 6 
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Descriptive Statistics 











Variable: C3l Wall 
Clock Time 
Anderson-Darling Normality Test 
A-Squarec: 1.158 
P-Value: 0.004 
Nean 94.5186 
StDev 3.2978 
Variance 10.8752 
Skewness 1.23957 
Kurtosis 1.19005 
N 30 
Mnimum 90.150 
1st Quartile 92.318 
Median 93.814 
3rd Quartile 95.862 
Vexmum 104.418 
95% Confidence Interval for Mu 
. : | 93.287 95.750 
$2 = % ‘i 95% Confidence Interval for Sigma 
ee cme $ cans ll 
— : 95% Confidence Interval for Median 
95% Confidence Interval for Median 92.670 04.652 
Descriptive Statistics 
Variable: FP Wall 
Clock Tire 
Anderson-Darling Normality Test 
A-Squarect 1.159 
P-Value: 0.004 
Mean 77.1691 
SDev 3.26/7 
Vanance 10.6780 
Skewness 1.25967 
Kurtosis 1.24295 
N <0. 
Mnimum 72.9853 
1st Quartile 75.0810 
Medan 76.4142 
3rd Quartile 78.5339 
Maximum 87.05/75 
95% Confidence Interval for Mu 
75.9489 78.3893 
95% Confidence Internal for Sgma 
2.6024 4.3028 
95% Confidence Interval for Vedian 
95% internal f 
Contidence Interval for Medan 753884 773701 
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Descriptive Statistics 





95% Confidence Interval for Mu 


rae y x Sas 
*) Parte enh al - V6 4 ora 
Sales =e, -, aut 
+ - * 








95% Confidence Interval for Vedian 


92.135 


2.604 


91.614 


Variable: 
Detect Wall 
Clock Time 
Anderson-Darling Normality Test 
A-Squared: 1219 
~ P-Value: 0.003 
Mean 93.3564 
StDev 3.2702 
Vaniance 10.6940 
Skewness 1.28097 
Kurtosis 1.29780 
30 
Mnimum 89.135 
1st Quartie 91.299 
NVedian 92.574 
3rd Quartile 94.672 
NVexmum 103.287 


95% Confidence Interval for Mu 


94.578 


95% Confidence Interval for Sgma 


4.396 


95% Confidence Interval for Median 


93.592 


Descriptive Statistics 
Variable: C3l IPC 





Bytes Read 
Anderson-Darling Normality Test 
A Squared: 0.698 
P-Value: 0.061 
Mean 166758 
StDev 8602 
Vaniance 73992428 
Skewness 6.34E-02 
' | Kurtosis -1.10990 
150000 156000 162000 168000 174000 180000 N 30 
, 7 . | | : Mnimum 149864 
—_ iad istQuarile 160522 
Median 164812 
3rd Quartile 174002 
Vexmum 179904 


95% Confidence Interval for Mu 





sae as 
_ 





5% Confidence Intenal for Median 


ew 


163546 


6851 


161100 


95% Confidence Interval for Mu 


169970 


95% Confidence Interval for Sgma 


11564 


95% Confidence interval for Median 


172524 


Descriptive Statistics 








Variable: FP IPC 
Bytes Read 
Anderson-Darling Normality Test 
A-Squared: 3.569 
P-Value: 0.000 
Mean 3782.13 
StDev 235.80 
Vaniance 55603.6 
Skewness 3.14058 
Kurtosis 10.7451 
N 30 
Mnimum 3584.00 
1st Quartile 3676.00 
Median 3728.00 
3rd Quartile 3808.00 
Nexmum 4832.00 
95% Confidence Interval for Mu 
| , 3694.08 3870.18 
3680 3780 SSO 95% Confidence Interval for Sigma 


Ee ed ee a 
95% Confidence Interval for Median 


95% Confidence Interval for Median 





3696.00 3790.85 
Descriptive Statistics 
Variable: Detect 
IPC Bytes Read 
Anderson-Darling Normality Test 
A-Squared: 0.890 
P-Value: 0.020 
Mean 78411.2 
StDev 4229.0 
Variance 17884811 
Skewness -2.8E-01 
Kurtosis 1.12307 
N 30 
Mnimum 67520.0 
1st Quartile 76630.0 
Median 78284.0 
3rd Quartile 80210.0 
Vexmum 88352.0 
95% Confidence Interval for Mu 
76832.0 79990.4 
ESS FON ii aa 95% Confidence Interval for Sigma 
ane 
95% Confidence Interval for Median SSCS Ue Fe M GN Nee En 

77516.1 79646.9 
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Descriptive Statistics 


Variable: C3l IPC 

Bytes Written 

Andersonm-Darling Normality Test 
A- Squared 0.870 
P-Value 0.022 
Mean 69327.5 
StDev 4128.7 
Vanance 17046254 
Skewness 1.64561 
Kurtosis 4.14283 
N 30 
Mnimum 64024.0 
1st Quartile 66364.0 
Median 68792.0 
3rd Quartile 70874.0 
Vexmum 84752.0 


677858 


3288.1 





95% Confidence Interval for Median 





67560.3 70250.0 
Descriptive Statistics 
Variable: FP IPC 
Bytes Written 

Andersorn-Darling Normality Test 
A-Squared: 0.369 
P-Value: 0.404 
Mean 2244479 
StDev 116675 
Vanance 1.36E+10 
Skewness -1 2E-01 
: . . : 7 Kurtosis ~47E-01 
2ooo0dd © 2100000 2200000 2300000 2400000 _ - 
: | : i Mnimum 1982960 
__ oo 
Median 2219934 
3rdQuartile 2322549 
Maximum 2460432 


95% Confidence Interval for Mu 


os 








95% Confidence Intervai for Median 


ee, 


2200912 


92921 


2180069 


95% Confidence Interval for Mu 


70869.2 


95% Confidence Interval for Sigma 


5550.3 


95% Confidence Interval for Median 


95% Confidence Interval for Mu 


2288046 


95% Confidence Interval for Sigma 


156848 


95% Confidence Internal for Median 


2304801 


Descriptive Statistics 
Variable: Detect IPC 








Bytes Written 
Anderson-Darling Normality Test 
A-Squared: 0.301 
P-Value: 0.557 
Mean 1358588 
StDev 73026 
Variance 5.33E+09 
Skewness 0.215106 
Kurtosis -9.0E-01 
N 30 
Mnimum 1225124 
1st Quartile 1302537 
NVedian 1350522 
3rd Quartile 1417316 
Vexmum 1502744 
95% Confidence Interval for Mu 
1331319 1385856 
95% Confidence Interval for Sigma 
58159 98170 
95% Confidence Interval for Median 
1312202 1399865 
Descriptive Statistics 
Variable: C3! IPC 
Number of Reads 
Anderson-Darling Normality Test 
ASquared: 0.698 
P-Value: 0.061 
Mean 20844.8 
StDev 10752 
Vanance 1156132 
Skewness 6.34602 
Kurtosis -1.10990 
N 30 
Mnimum 18733.0 
1st Quartile 20065.3 
Median 20601.5 
3rd Quartile 21750.3 
Vaxmum 22488.0 
95% Confidence Interval for Mu 
20443.3 21246.3 
95% Confidence Interval for Sigma 
856.3 1445.5 
95% Confidence Interval for Vedian 
5% » f, - 
95% Confidence Interval for Median 201375 >15655 


140 


Descriptive Statistics 











Variable: FP IPC 
Number of Reads 
Anderson-Darling Normality Test 
_ ASquared: 3.569 
P-Value: 0.000 
Nean 472.767 
StDev 29.476 
Variance 868.806 
Skewness 3.14058 
| : Kurtosis 10.7451 
£5) £90 530 5/0 610 * 2 
H H 
Mnimum 448.000 
1st Quartle 459.500 
NVedian 466.000 
3rd Quartile 476.000 
Vexmum 604.000 
95% Confidence Interval for Mu 
. 3 , 461.760 483.773 
460 465 470 475 ae — 95% Confidence Interval for Sigma 
23.475 39.624 
= e 95% Confidence Interval for Median 
% Confidence intenal for Vz 

nee interna! for Median 462.000 473.856 


Descriptive Statistics 
Variable: Detect IPC 





Number of Reads 
Anderson-Darling Normality Test 
A-Squared: 0.890 
P-Value: 0.020 
Mean 9801.40 
SiDev 528.63 
Vanance 279450 
Skewness -2.8E-01 
Kurtosis 1.12307 
N 30 
Mnimum 8440.0 
1st Quartile 9578.7 
Median 9785.5 
3rd Quartile 10026.2 
MVaxmum 11044.0 
95% Confidence Interval for Mu 
9604.0 9998.8 
95% Confidence Interval for Sigma 
421.0 710.6 
©5%% Confidence Intena! for Median S572 CORN Gt Micival OT EGEN 
9689.5 9955.9 
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Descriptive Statistics 














Variable: Cel IPC 
Number of Writes 
Anderson-Darling Normality Test 
A-Squared: 3.439 
P-Value: 0.000 
Mean 1077.30 
StDev 63.03 
Vaniance 3972.98 
Skewness 3.09582 
7 ; ; : Kurtosis 10.6610 
1000 1100 1200 1300 N 30 
. | Mnimum 1022.00 
i : ° 1stQuarile 1052.75 
Median 1062.50 
3rd Quartile 1083.00 
95% Confidence Interval for Mu Mexmum 1358.00 
CS a i ea "e 95% Confidence Interval for Mu 
: , , ; 1053.76 1100.84 
iGO 1000 10%) 1080 1090 1100 95% Confidence Interval for Sigma 
! | 
: 50.20 84.73 
ae 95% Confidence Interval for Median 
5% we rm § 
% Confidence Interval for Median 1055.00 1078.77 
Descriptive Statistics 
Variable: FP IPC 
Number of Writes 
Anderson-Darling Normality Test 
A-Squared: 0.353 
P-Value: 0.443 
Mean 59372.4 
StDev 3059.6 
Variance 9361193 
Skewness -1.4E-01 
Kurtosis -3.8E-01 
N 30 
Mnimum 52426.0 
1st Quartile 57507.0 
Nedian 58784.0 
3rd Quartile 61376.0 
Vexmum 65144.0 
95% Confidence Interval for Mu 
58229.9 60514.9 
95% Confidence Interval for Sgqma 
2436.7 4113.1 
95% Confidence Interval for Median 
5% Interval f : 
95% Confidence Interval for Vedian 577429 60958.0 
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Descriptive Statistics 








Variable: Col 
Network Bytes Written 
Anderson-Darling Normality Test 
A Squared: 1.049 
P-Value: 0.008 
Mean 1634463 
StDev 64789 
Vanance 4.20E+09 
Skewness 1.32862 
| ! ! : ' — Kurtosis 2.16355 
1550000 1 600000 1 850000 77000001 750000 1800000 1850000 N 30 
' , ) | | : ! Mnimum 1541245 
—_iiy_— “4 1st Quartile 1598278 
Median 1621830 
3rd Quartile 1661383 
95% Confidence Interval for Mu Vexmum 1851427 
} 95% Confidence Interval for Mu 
: 3 : | = ) 1610270 1658656 
ig ecg ote Moa org 95% Confidence Interval for Sigma 
| l 
51598 87097 


; 
x 
es 

ee 


5 


95% Confidence Intena! for Vedi 
= 1611896 


Descriptive Statistics 


95% Confidence Interval for Median 


1637375 


Variable: FP Network 





Bytes Written 
Anderson-Darling Normality Test 
ASquared: 0.709 
P-Value: 0.057 
Mean 1029378 
StDev 34171 
Variance 1.17E+09 
Skewness 0.273428 
Kurtosis 0.365661 
N 30 
Mnimum 955221 
1st Quartile 1011185 
NVedian 1025848 
3rd Quartile 1046070 
1108747 


95°, Confidence interval for Mu Vaxmum 


peeilig 


. ; , 1016618 


1042137 


1015000 "025000 es TO4SO00 95% Confidence Interval for Sgma 





27214 


5% Confidence interval for Median 
1013469 
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45936 


95% Confidence Interval for Median 


1038536 


Descriptive Statistics 


Variable: Detect 








Network Bytes Written 
Anderson-Darling Normality Test 
ASquared: 0.982 
P-Value: 0.012 
Mean 2057529 
S{Dev 68161 
Variance 4.65E+09 
Skewness -2.3E-01 
—s : ) | : Kurtosis 1.22879 
+900000 1 9s000020000002050000210000021500002200000 N 30 
| | ! Mnimum 1885618 
- — ft * —___ ° {stQuarile 2081778 
Median 2056892 
3rd Quartile 2090429 
95% Confidence Intenal for Mu ven cae 
ae. cages 95% Confidence Interval for Mu 
| 7 | : = 7 , 2032078 2082981 
zoesno) 2o8e000 2séc0 2068000 2n68000 zo7e0N0 2088000 95% Confidence Interval for Sigma 


pee . 4% Dat 


54284 





Confidence Interval for Vedian 


Fie on oe 
95% 





2038369 2074922 
Descriptive Statistics 

Variable: Col 

Network Number of 

Writes 

Anderson-Darling Normality Test 
A-Squarec: 0.733 
P-Value: 0.050 
Mean 155957 
StDev 7568 
Variance 57276532 
Skewness 1.41476 
Kurtosis 2.73825 

30 

Mnimum 145958 
1st Quartle 150646 
Median 154280 
3rd Quartile 159340 
Vexmum 182493 


95% Confidence Interval 


for Mu 





153131 
I 1 I | } i 
152000 153000 154000 155000 1560000 157000 158000 159000 
| 


% Confidence Inierval for Median 


6027 


152574 
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91630 


95% Confidence Interval for Median 


95% Confidence Interval for Mu 


158783 


95% Confidence Interval for Sigma 


10174 


95% Confidence Interval for Median 


158076 


Descriptive Statistics 
Variable: FP Network 





Number of Writes 

Anderson-Darling Normality Test 

A-Squared: 0.561 

P-Value: 0.134 

Mean 741.300 

StDev 1.705 

Variance 2.90690 

Skewness 0.150117 

Kurtosis -7.2E-01 

N 66 

Mnimum 738.000 

1st Quartile 740,000 

Median 741.000 

3rd Quartile 743.000 

Maximum 745.000 

95% Confidence Interval for Mu 

; 740.663 741.937 
pe a i Fa 95% Confidence Interval for Sigma 
OB), Goniidenceltiane omen 95% Confidence Interval for Median 

—_—, 741.000 742.000 


Descriptive Statistics 





Variable: Detect 
Network Number of 
Writes 
Anderson-Darling Normality Test 
A-Squared: 1.265 
P-Value: 0.002 
Mean 588.933 
StDev 8.034 
Variance 64.5471 
Skewness -3.3E-01 
Kurtosis 1.49254 
N 30 
Mnimum 568.000 
1st Quartile 586.000 
Median 589.000 
3rd Quartile 592.250 
Vexmum 608.000 
95% Confidence Interval for Mu 
585.933 591.933 
95% Confidence Interval for Sigma 
6.398 10.800 
95% Confidence Interval for Median 
52%, fidence Internal for Media: 
95% Confidence inte or ian 587.000 590.771 
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Descriptive Statistics 











Variable: C3 
Cumulative CPU Times 
(C3i.pstat) 
Anderson-Darling Normality Test 
A-Squared: 0.200 
P-Value: 0.871 
Mean 20.7333 
StDev 0.8770 
Variance 0.769195 
Skewness 0.324038 
Kurtosis -2.3E-01 
N 30 
Mnimum 18.9700 
1st Quartile 20.1500 
Median 20.7100 
3rd Quartile 21.3925 
Veaxmum 22.8600 
95% Confidence Interval for Mu 
! | | | a algee 
| q j l 
| a 0.6985 1.1780 
— 95% Confidence Interval for Median 
% , 
95% Confidence Interval for Median 90.2491 91.0440 
Descriptive Statistics 
Variable: FP 
Cumulative CPU Times 
(fp.pstat) 
Anderson-Darling Normality Test 
A-Squared: 0.308 
P-Value: 0.541 
Mean 20.3123 
StDev 0.9480 
Variance 0.898646 
Skewness 0.232861 
Kurtosis 1.10592 
N 30 
Mrnimum 17.9300 
1st Quartile 19.7825 
NVedian 20.3450 
3rd Quartile 20.8250 
95% Confidence Interval for Mz Ven aoa 
[to aa ahaa 
Cp oo 19.9584 20.6663 
19.25 1995 OG 215 DB OS NS OS ONS VS 95% Confidence Interval for Sigma 
| | | | | 


4 


| on Se | a age 
95% Confidence Interval for Median 


95% Confidence Interval for Median “es 20 5031 
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Descriptive Statistics 





Variable: Detect 
Cumulative CPU 
Times (detect.pstat) 
Anderson-Darling Normality Test 
P-Value: 0.123 
Mean 22.1503 
otDev 1.1457 
Variance 1.31253 
Skewness 0.668464 
Kurtosis -2.6E-02 
N 30 
Mnimum 20.0300 
1st Quartile 21.3250 
Median 22.0000 
3rd Quartile 22.9225 
Vaexmum 25.1600 
95% Confidence Interval for Mu 
21.7225 22.5781 
95% Confidence Internal for Sigma 
0.9124 1.5401 
95% Confidence Interval for Median 
o/ - I 
95% Confidence Interval for Vedian 21.3869 2? 4326 


Descriptive Statistics 


Variable: C3l Physical 
Pages in Memory 











Anderson-Darling Normality Test 
A-Squared: 2.113 
P-Value: 0.000 
Mean 2953.47 
otDev 0.78 
Variance 0.602299 
Skewness -1.1E-01 
Kurtosis -5.6E-01 
N 30 
Mnimum 2052.00 
1st Quartile 2953.00 
Median 2953.50 
3rd Quantile 2954.00 
nce Interval for Mu Vexmum 2955.00 
Ep RP eG 95% Confidence Interval for Mu 
} ——— | 2953.18 2953.76 
aoe ae? 2954.0 95% Confidence Interval for Sigma 
; 0.62 1.04 
Ts ecg : 95% Confidence Intenal for Median 
95% Confidence interval for Median 
seca 2953.00 2954.00 
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Descriptive Statistics 
Variable: C3l Pages in 





Virtual Memory 

Anderson-Darling Normality Test 

A-Squared: 2.113 

P-Value: 0.000 

Mean 23627.7 

StDev 6.2 

Vanance 38.5471 

Skewness -1.1E-01 

Kurtosis -5.6E-01 

N 30 

Mnimum 23616.0 

1st Quartile 23624.0 

Median 23628.0 

3rd Quartile 23632.0 

Veximum 23640.0 

95% Confidence Interval for Mu 

23625.4 23630.1 
: : ; 95% Confidence Interval for Sigma 

i i 
95% Confidence Interval for Median 

% Confidence | fi 
95% Confidence Interval for Median 40 0 


Descriptive Statistics 
Variable: C3l Pages in 


Resident Set 
Anderson-Darling Normality Test 
A-Squared: 1.148 
—— P-Value: 0.004 
Mean 17964.8 
StDev 11.8 
Variance 139.476 
Skewness -7.5E-01 
Kurtosis 0.361664 
N 30 
Mnimum 17936.0 
1st Quartile 17960.0 
Median 17968.0 
3rd Quartile 17976.0 
Vexmum 17984.0 


95% Confidence interval for Vedan 
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17960.4 


9.4 


17960.0 


95% Confidence Interval for Mu 


17969.2 


95% Confidence Interval for Sigma 


15.9 


95% Confidence Interval for Median 


17968.0 


Descriptive Statistics 
Variable: FP Pages in 








Resident Set 
Anderson-Darling Normality Test 
A-Squared: 11.090 
P-Value: 0.000 
Mean 7911.73 
StDev 1.46 
Variance 2.13333 
Skewness ~4.94167 
| : ; | Kurtosis 23.1967 
7004 706 7208 7310 7312 N 30 
! | 
| , Mnimum 7904.00 
° 1st Quartile 7912.00 
Vedian 7912.00 
3rd Quartile 7912.00 
NVexmum 7912.00 
95% Confidence Interval for Mu 
| ; 7911.19 7912.28 
M12 AN? M22 95% Confidence Interval for Sigma 
| | 
1.16 1.96 
95% Confidence Interval for Median 
05% ff oy Ir & | for 
5% Confidence Interval for Median 7912.00 7912.00 
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APPENDIX I. MSHN WRAPPER OUTPUT: DETERMINISTIC TRIALS 


Note: Only data that generated a distribution is displayed. 


Descriptive Statistics 
Variable: C3l User 


CPU Time 

(Deterministic) 

Anderson-Darling Normality Test 
A-Squared: 1.433 
P-Value: 0.001 
Mean 34.3613 
StDev 0.6598 
Variance 0.435329 
Skewness 1.18541 
Kurtosis 0.564623 
N 30 
Mnimum 33.4900 
1st Quartile 33.9200 
Median 34.1750 
3rd Quartile 34.5775 
Vaximum 36.0600 

95% Confidence Interval for Mu 

34.1150 34.6077 

95% Confidence Interval for Sigma 
0.5255 0.8870 





QE 9°. Crnficlancea Intanal far Martian 


Descriptive Statistics 
Variable: FP System 








CPU Time 
(Deterministic) 
Andersor-Darling Normality Test 
A-Squared: 0.380 
P-Value: 0.381 
Mean 4.35667 
StDev 0.11989 
Vanance 1.44E-02 
Skewness 0.389254 
Kurtosis -3.3E-01 
N 30 
Mnimum 4.12000 
1st Quartile 4.27500 
Nedian 4.35000 
3rd Quartile 4.41500 
MVeximum 4.65000 
95% Confidence Interval for Mu 
4.31190 440144 
95% Confidence Interval for Sigma 
0.09548 0.16118 
95% Confidence interval for Median Pome eo eal OnE aN 
4.31000 4.39543 
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Descriptive Statistics 








Variable: Detect 
system CPU Time 
(Deterministic) 
Anderson-Darling Normality Test 
A-Squared: 0.407 
P-Value: 0.328 
Mean 11.1957 
StDev 0.3933 
Variance 0.154687 
Skewness 0.480020 
Kurtosis -5.8E-01 
N 30 
Mnimum 10.6500 
ist Quartle 10.8675 
Median 11.2200 
3rd Quartile 11.3875 
NVaxmum 12.0600 
95% Confidence Interval for Mu 
11.0488 11.3425 
; ; 95% Confidence Interval for Sigma 
eae 0.3132 0.5287 
Saal neaiantiaaaitae 95% Confidence Intenal for Median 
95% Confidence Interval for Median 1D) a 


Descriptive Statistics 
Variable: Col System 





CPU Time 
(Deterministic) 
Anderson- Darling Normality Test 
A-Squared: 0.393 
P-Value: 0.356 
Mean 5.07300 
StDev 021459 
Variance 4,.60E-02 
Skewness -3.0E-01 
Kurtosis ~82E-01 
N 30 
Mnimum 4.64000 
1st Quartile 4.89750 
Median 5.11000 
3rd Quartile 5 23500 
Nexmum 5.46000 
95% Confidence Interval for Mu 
4.99287 5.15313 
95% Confidence Interval for Sigma 
0.17090 0.28848 
95% Confidence Interval for Median 
5% Cont inter 
95% Confidence Interval! for Median A 501314 
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Descriptive Statistics 
Variable: FP System 














CPU Time 
(Deterministic) 
Anderson-Darling Normality Test 
A-Squared: 0.380 
P-Value: 0.381 
Mean 4.35667 
StDev 0.11989 
Variance 1.44E-02 
Skewness 0.389254 
Kurtosis -3.3E01 
N 30 
Mnimum 4.12000 
1st Quartile 427500 
Median 4.35000 
3rd Quartile 4.41500 
Vexmum 4.65000 
95% Confidence Interval for Mu 
4.31190 440144 
95% Confidence Interval for Sigma 
0.09548 0.16118 
05% Confidence Internal for Vecdian S70 Corse Ninel On AaT 
4.31000 4.39543 
Descriptive Statistics 
Variable: Detect 
System CPU Time 
(Deterministic) 
Anderson-Darling Normality Test 
A-Squared: 0.407 
P-Value: 0.328 
Mean 11.1957 
StDev 0.3933 
Vanance 0.154687 
Skewness 0.480020 
Kurtosis -5.8E-01 
N 30 
Mnimum 10.6500 
1st Quartile 10.8675 
Nedian 112200 
3rd Quartile 11.3875 
95% Confidence Intenal for Mu Vaxmum 12.0600 
ee = 4 95% Confidence Interval for Mu 
) - — 11.0488 11.3425 
Wes nS 114.15 11.25 11.35 95% Confidence Interval for Sigma 


Sa t= oor, 


Z , 
95% Confidence interval for Median Be EONS eda rata 
10.9483 11.3386 


13 


Descriptive Statistics 








Variable: C3l Wall 
Clock Time 
(Deterministic) 
Anderson-Darling Normality Test 
A-Squared: 1.104 
P-Value: 0.006 
Mean 141.020 
StDev 1.107 
Variance 1.22540 
Skewness 1.45440 
Kurtosis 2.30142 
N 30 
Minimum 139.770 
1st Quartile 140.099 
Median 140.840 
3rd Quartile 141.416 
Vexmum 144.761 
95% Confidence Interval for Mu 
140.606 141.433 
95% Confidence Interval for Sigma 
= — 0.882 1.488 
—_— SS 95% Confidence Interval for Median 
5% rf { 
95% Confidence Interval for Median 440.247 141.289 


Descriptive Statistics 





Variable: FP Wall 
Clock Time 
(Determinstic) 
Anderson-Darling Normality Test 
A-Squared: 1.050 
P-Value: 0.008 
Nean 124.497 
StDev 1.108 
Vanance 1.22803 
Skewness 1.35971 
Kurtosis 1.89990 
N 30 
Mnimum 123.244 
1st Quartle 123.584 
Nedian 124.309 
3rd Quartile 124.885 
Vexmum 128.139 
95% Confidence Interval for Mu 
124.083 124.911 
95% Confidence Interval for Sgma 
0.883 1.490 
95% Confidence Interval for Median 
5% Con | for Mec 
05% Confidence Interval fo ian 123.608 124.746 
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Descriptive Statistics Wanables pciecenvall 








Clock Time 
(Deterministic) 
Anderson-Darling Normality Test 
A-Squared: 1.120 
P-Value: 0.005 
Mean 140.808 
StDev 1.130 
Vaniance 1.27736 
Skewness 1.43378 
Kurtosis 2.12833 
N 30 
Mnimum 139,579 
1st Quartile 139.885 
Nedian 140.600 
3rd Quartile 141.175 
Vexmum 144.573 
95% Confidence Interval for Mu 
140.386 141.230 
95% Confidence Interval for Sigma 
: : 0.900 1.519 
ah 95% Confidence Interval for Median 
% fi + | ? : 
95% Confidence Interval for Median 440.085 141.046 
Descriptive Statistics __. 
Variable: Col 
Cumuative CPU Time 
(ci. pstat) 
Anderson-Darling Normality Test 
A-Squared: 2.236 
P-Value: 0.000 
Mean 39.3937 
StDev 0.7044 
Variance 0.496127 
Skewness 1.07294 
Kurtosis 0.315679 
N 30 
Mnimum 38.1800 
1st Quartile 38.9925 
Median 39.1150 
3rd Quartile 39.6700 
Vexmum 41.1200 





5 306 307 


83 4 
! | ! ! 





05°% Confidence Interval for Vedian 
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39.1307 


0.5610 


39.0100 


95% Confidence Intervel for Mu 


39.6567 


95% Confidence Interval for Sgma 


0.9469 


95% Confidence Interval for Median 


39.3400 


Descriptive Statistics 











Variable: FP 
Cumulative CPU Time 
(fp.pstat) 
Anderson-Darling Normality Test 
A-Squared: 5.549 
P-Value: 0.000 
Mean 24.8813 
StDev 12281 
Vanance 1.50816 
Skewness 429112 
? ; Kurtosis 18.8038 
26 xs 28 30 N 30 
| Mnimum 24.2200 
-}- : ° 4st Quartile 94.4250 
Median 24.5400 
3rd Quartile 24.8925 
% Confidence Interval for Mu Maximum 31.0900 
ny 95% Confidence Interval for Mu 
| — ) 24,4228 25 3309 
24.4 oN a 95% Confidence Interval for Sigma 
A — a 
95% Confidence Interval for Median 
=O/ 
95% Confidence Interval for Median 24,4423 4.7577 
Descriptive Statistics 
Variable: Detect 
Cumulative CPU Time 
(detect.pstat) 
Anderson-Darling Normality Test 
A-Squared: 1.014 
P-Value: 0.010 
Mean 38.2990 
StDev 0.3719 
Variance 0.138306 
Skewness 0.907432 
Kurtosis -1.1E01 
N 30 
Mnimum 37.7900 
1st Quartile 38.0375 
Median 38.1800 
3rd Quartile 38.5175 
Vexmum 39.1700 


5% Confidence Interval for Mu 


Fe hient 
ee rn 
© 4 
Pee ene 
KA ee r 
Ree eee 
‘a 








95% Confidence Interval for Median 


156 


38.1601 


0.2962 


38.1014 


95% Confidence Interval for Mu 


38.4379 


95% Confidence Interval for Sigma 


0.4999 


95% Confidence Interval for Median 


38.3631 
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SSPK Single Shot PK 
Route PK Total PK For Killing Target 
Time Target Engager Range Az 
t SSPK PKCum Route 
(Seconds ) ID ID (m) (Deg) 

(Deg) PK 
a ee eee ee esc emcees es ec ammtct cers es ome ce cen aera) cme tome Fm Cacme’ Sessa due Ne 7 SY aC mm Fa Goa | as ae mS mm ee meen aan Pome ceca fame «) am acm” amas es cae ame) one 

104.96 Missile 28767 Asset_Def_SAM_03 ii 2a 2s 0 ace 230095 
80.00 80.00 80.00 

105.52 Missile 28766 Asset_Def_SAM_01 1S4i09 12598." v23734 
80.00 80.00 80.00 

106.89 Missile 28768 Asset_Def_SAM_01 19006 8.33 24.225 
80.00 80.00 80.00 

107.83 Missile 28767 Asset_Def_SAM_03 15669 32.85 31.13 
80.00 96.00 96.00 

108.03 Missile 28766 Asset_Def_SAM_01 16619 914035 © 22.54 
80.00 96.00 96.00 

109.74 Missile 28768 Asset_Def_SAM_01 17400 9.64 23.61 
80.00 96.00 96.00 

110.42 Hostile_AA_Ftr_04 Point_Def_SAM_01/03 4698 TAZ 2 224.564 
50.00 50.00 50.00 

118.45 Hostile_AA_Ftr_04 Point_Def_SAM_01/03 2621s 2 Seo a Aea 38 
5000. 575.00: 175.00 

119.54 Hostile_AA Ftr_06 Point_Def_SAM_01/03 3912 357.86 29.11 
50.00 50.00 -=S0a00 

120.30 Missile 28767 Asset_Def_SAM_03 8526 51.85 24.06 
80.00 99.20 99220 

121.18 Missile. 28768 Asset_Def_SAM_03 VihGo Waa 40 25674 
80.00 99.20 99:20 

125.23 Hostile_AA Ftr_05 Point_Def_SAM_01/03 4161 76.62 27.29 
50.00 50.00 50.00 

127.88 Hostile_AA Ftr_06 Point_Def_SAM_01/03 2087 354.08 65.24 
50-00 975.00). 7500 

128.33 Hostile_AA_Ftr_04 Point_Def_SAM_01/03 2156 157.01 64.40 
50.00 87.50 87.50 

132240 oMissite 20773 Asset_Def_SAM_01 184109. 012.98 <0 323234 
80.00 80.00 80.00 

132.48 Missile 28774 Asset_Def_SAM_03 15669 32.85 31.13 
80.00 80.00 80.00 

134.01. Missile 28775 Asset_Def_SAM_01 19006 B33 aes 
80.00 80.00 80.00 

134.47 Hostile_AA_Ftr_07 Point_Def_SAM_01/03 3823 9.04 230277 
50.00 50.007 50.00 

135.11 Missile=28773 Asset_Def_SAM_01 16819 14.03 22.54 


80.00 96.00 96.00 —_ 
135.19 Missile. 28774 Asset_Def_SAM_03 14106 35.50 30.98 


s 


136.82 Missile 28775 Asset_Def_SAM_01 17400 9.64 23.61 
80.00 96.00 96.00 

142.93 Hostile_AA_Ftr_07 Point_Def_SAM_01/03 2103, BIS 67229 
50.00 75.007 #7500 

147.77 Missile 28775 Asset_Def_SAM_03 77460 .. 47.40 (“25.74 


80.00 99.20 99.26 
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Single Shot PK 
PKCum Cumulative PK Of Individual Shots 
Route PK Total PK For Killing Target 
Time Target Engager Range Az 
1 Solk PKCum Route 
(Seconds) ID) ID (m) {Deg) 

(Deg) PK 
nS eo oo a Oe oo oo oo SS oS oe Soo OO oo SS Sa ae Ss SS = 

155.10 Red_AG Ftr_01/04 Point_Def_SAM_01/01 335neee 1 599 Oats: 
50.00 50200 50-00 

155.53 Red_AG_Ftr_01/03 Point_Def_SAM_01/01 42710 113.46 Onsz 
50-00 50.00 50.00 

156.97 Red_AG Ftr_01/01 Point_Def_SAM_01/01 S650 ae be oewae 0.40 
50.00 50200 50.00 

158.05 Hostile_AA Ftr_02/01 Point_Def_SAM_01/03 6901 ey oe Oa 15252 
42.16 42.16 42.16 

158.13 Hostile_AA_Ftr_02/03 Point_Def_SAM_01/03 6980 42.32 16.20 
40.93 40.93 40.93 

158.13 Hostile_AA_Ftr_02/04 Point_Def_SAM_01/03 6980 A232 16220 
40.93 40.93 40.93 

161.66 Missile 28777 Asset_Def_SAM_03 ge 2 30.60 30295 
80.00 80.00 80.00 

162.51 Missile 28776 Asset_Def_SAM_01 18410 12.968 23558 
80.00 80.00 80.00 

163.89 Missile 28778 Asset_Def_SAM_01 19006 8.33 24225 
80.00 80.00 80.00 

164.56 Missile 28777 Asset_Def_SAM_03 15669 32.85 312s 
80.00 96.00 96.00 

165.03 Missile 28776 Asset_Def_SAM_01 16819 14.03 22.54 
80.00 96.00 96.09 

165.70 Red_AG Ftr_02/03 Point_Def_SAM_01/01 a5 21207 4 0.98 
50.00 50200 50.00 

166.62 Red_AG Ftr_01/04 Point_Def_SAM_01/01 eels 137.52 1.42 
50.00 75200 75.00 

166.74 Missile 28778 Asset_Def_SAM_01 17400 9.64 23 .6u 
80.00 96.00 96.00 
166.97 Red_AG Ftr_01/03 Point_Def_SAM_01/01 548475 5134.33 1.72 
50.00 75-00 howe O18) 

169.44 Hostile_AA Ftr_05 AA_Ftr_04/01 9129 324.81 3545238 
70.00 85.00 85.00 

170.07 Hostile_AA Ftr_05 AA_Ftr_04/02 9305-)325.233 352208 
70.00 95250 95.50 

170.41 Hostile_AA Ftr_02/01 Point_Def_SAM_01/03 4576 38.66 232.35 
50.00 Tig OS 70s 

170.53 Hostile_AA_ Ftr_02/03 Point_Def_SAM 01/03 4693 46e7 1 24.55 
50.00 70.47 AoW? 

170.53 Hostile_AA Ftr_02/04 Point_Def_SAM_01/03 4693 46.71 24255 
S0200 10.47 70.47 

171. 83°) BASE Sr 204701 Hostile _AA Ftr_05 Ssog4 133.61 BerZ 
45.00 45.00 45.00 

173.49 AA_Ftr_04/01 Hostile AA Ftr_04 7910 138.02 354.245 
45.00 69.75 48.09 

176.86 Missile 28777 Asset_Def_SAM_03 9782 46.69 27.23 
80.00 99.20 99.29 

177.92 Missile 28778 Asset_Def_SAM_03 7746 “Ag-40- 25294 
80.00 99.20 99.20 
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PAPA Report 


Scenario: Demo300 Report generated on Thu May 6 07:43:10 1999 


Report Type: PRK HISTORY 


PAPA PK Legend 


cme eee eee ee 


300% 


0. 


69 


94 


SSPK Single Shot PK 
PKCum Cumulative PK Of Individual Shots 
Route PK Total PK For Killing Target 
Time Target Engager Range Az 
i SSPK PKCum Route 
(Seconds) ID ID (m) (Deg) 
(Deg) PK 
le SSso SSS ae a ee a ae ee aS See eS a Sra SS ea Sea SS eee Be eee aa SaaS Secret Sees ea Sas. Cae eS eae 
178241 AAFer.04/02 Hostile_AA Ftr_04 D1 See ole 
45.00 45.00 45.00 
178.62 Red_AG_Ftr_01/04 Point_Def_SAM_01/01 6992 149.19 
40. 75 35.19 85.19 
178.88 Hostile_AA Ftr_02/02 Point_Def_SAM_01/03 3052 20.207 
39.78 50500 50200 50.00 
180.18 AA _Ftr_04/01 Hostile_AA_Ftr_06 440% 7124.33 
B45207 45.00 Sane 53.93 
183.18 Red_AG Ftr_02/03 Point_Def_SAM_01/01 6818 144.39 
i oe 43.46 oe es 71.73 
183.32 — AA_FPEr_04/02 Hostile_AA Ftr_06 62370r 130-46 
346.44 45.00 69.75 51.19 
187.82 AA_Fer_04/01 Hostile_AA Ftr_07 9445 129.68 
354.04 45.00 90-85 SiS a, 
191.44 AA Ftr_04/02 Hostile AA Ftr_07 9307" 137265 
356.55 45.00 9523.6 56.68 
192.35 Missile 28785 Asset_Def_SAM_03 15669 32-6) 
ech ae 80.00 80.00 80.00 
192.48 Missile 28784 Asset_Def_SAM_01 18410 12-98 
Zon 04 80.00 80.00 80.00 
193.86 Missile 28786 Asset_Def_SAM_01 19006 8.33 
24.25 80.00 80.00 80.00 
195.01 Missile 28784 Asset_Def_SAM_01 16819 14.03 
22.54 80.00 96.00 96:00 
195.10 Missiie 29785 Asset_Def_SAM_03 14106 35.50 
30.98 80.00 96.00 96:06 
195.90 Red_cM_01 AA_Ftr_03/02 9469 349.78 
Bo4. 20 70.00 70.00 70.00 
196.91 Missile 28786 Asset_Def_SAM_01 17400 9.64 
VR gol 80.00 96.00 96.00 
197.3) Hostile AACr cr 204 AA_Ftr_03/01 10731 19. 30 
344.55 70.00 96.25 96.25 
198.57 Hostile_AA Ftr_04 BA Fer 05/01 17463 “291282 
349.52 $8.93 98.46 98.46 
205.73. RACPer205702 Hostile _AA Ftr_06 6252 PlOls37 
358.92 A500 45.00 Ace 0.0 
208.50 Missile 28786 Asset_Def_SAM_03 FIAG AT AO 
25.74 80.00 99.20 99.20 
209.11 HostilesnAAarrr 03/03 Point_Def_SAM_01/02 7640 287.85 
16.56 30.61 30.61 30061 
209.11 Hostile_AA Ftr_03/04 Point_Def_SAM_01/02 7640 287.85 
16.56 30,61 50.61 30.61 
209.23 Red_cCM_01 AA Fer 03/01 S150 325.00 
320.89 70.00 51200 2100 
211.48 Hostile_AA Ftr_07 AA Fer 04/01 aco. 356222 
359.50 70.00 92.50 S215 
217.46 Red_CM_03 AA _Ftr_03/02 6644 298.96 
342.16 70.00 702.00 70.00 
217.53 Red_CM_04 AA_Ftr_03/02 4367 287.11 
331.46 79200 70.00 70.00 
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PAPA Report 
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SSPK Single Shot PK 
PKCum Cumulative PK Of Individual Shots 
Route PK Total PK For Killing Target 
Time Target Engager 
i SSPK PKCum Route 
(Seconds) LD ID 
(Deg) PK 
LaSaSe Se oee aso 
220.41 Hostile_AA Ftr_06 PASE er 05701 
a4 1.30 FO 200 Be 50 92.50 
220.86 AA Ftr_03/01 Hostile_AA Ftr_04 
14.229 45.00 45.00 45.00 
221.25 Missile 28787 Asset_Def_SAM_01 
23.79 80.00 80.00 80.00 
221.43 AA _Ftr_04/01 Hostile_AA Ftr_05 
35906 45.00 94.97 59.94 
221.76 Missile 28788 Asset_Def_SAM 03 
30395 80.00 80.00 80.00 
222.96 Missile 28789 Asset_Def_SAM_01 
24225 80.00 80.00 80.00 
223.42 Hostile_AA Ftr_03/03 Point_Def_SAM_01/02 
6 oi PAG rab 49.58 23538 
223.42 Hostile_AA_Ftr_03/04 Point_Def_SAM_01/02 
ji{oyead el 274 49.58 49.58 
224.18 AA_Ftr_04/02 Hostile _AA Ftr_05 
S59201 45.00 90.85 57.50 
224.28 Missile 28787 Asset_Def_SAM_01 
23 334 80.00 96.00 96.00 
224.31 Hostile_SAM_Killer_01 Asset_Def_SAM_03 
i ealts, 80.00 80.00 80.00 
224.67 Missile 28788 Asset_Def_SAM_03 
Ses 80.00 96.00 96.00 
225.85 Missile 28789 Asset_Def_SAM 01 
23.61 80.00 96.00 96.00 
225.90 Red CM 02 AA_Ftr_03/02 
344.93 70.00 70.00 70.00 
226.13 AA_Ftr_04/01 Hostile AA Ftr_03/01 
348.90 45.00 O23 Tt.97 
227 Lo eee ey 04/01 Hostile_AA Ftr_03/02 
348.47 45.00 98.48 87.88 
227227 peer cr- 04/01 Hostile_AA Ftr_03/03 
348.40 45.00 99.16 90.63 
227227 AALS EEE 04/01 Hostile_AA Ftr_03/04 
348.40 45.00 99.54 92276 
231.64 Hostile _AA Frr_05 AA_Ftr_05/01 
344.61 70.00 98.65 98.65 
235.47 Hostile_AA Ftr_04 AA _Ftr_03/02 
3360-67 710 00 99.54 99.54 
235.48 Missile 28787 Asset_Def_SAM_03 
26.40 80.00 99.20 99726 
237.93 Missile 28788 Asset_Def_SAM_03 
24.06 80.00 99.20 eae 
238.82 Hostile_AA _Ftr_07 AA_Ftr_05/01 
S45 461-5 10.00 a > 94.65 
244.37 Hostile_AA Ftr_05 AA_Ftr_04/02 
35.38.55 70.00 99.60 99.05 
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Scenario: 


Report Type: 


Demo300 


PK. HISTORY 


PAPA PK Legend 


PAPA Report 


Report generated on Thu May 


Cumulative PK Of Individual Shots 


cme wie i i i i ee wee eee ee i i ee ee ee ee ew ew ew eS ae @mw ww ew =e — = — ee ——_— — — 


Engager 


BB, 


Hostile_AA_Ftr_02/01 
Hostile_AA Ftr_02/02 
Hostile_AA_Ftr_02/03 
Hostile_AA Ftr_02/04 
ESeort 02/01 
Escort_02/02 
AA_Ftr_05/02 
AA_Ftr_05/02 
AA_Ftr_04/01 
Asset_Def_SAM_03 
Asset_Def_SAM_03 
AA_Ftr_04/01 
AA_Ftr_05/01 
AA_Ftr_04/01 
AA_Ftr_05/01 
Escort_02/02 
Asset_Def_SAM_03 
Hostile_AA Ftr_04 
Escort_02/01 
Point_Def_SAM_01/03 
Asset_Def_SAM_03 
Hostile_AA_Ftr_02/01 
Ra aeer 03701 


AA Ftr_05/01 


SSPK Single Shot PK 
PKCum 
Route PK Total PK For Killing Target 
Time Target 
if SSPK PKCum Route 
(Seconds) ID 
(Deg) PK 
SS SS 
245.52. AA-Ftr_03/01 
350212 45.00 69'.75 52.16 
246000 AA Etre 03701 
348.39 45.00 B3.50 62.92 
246.75  AALFtr_03701 
348.25 45.00 90.85 67.85 
246.75 AALlFtr_03/01 
348.25 45.00 94.97 T1222 
250.74 Red_CM_01 
294.46 702100 97.30 97.30 
251.27 Red_CM_01 
298.60 70.00 99.19 99.19 
256.51 Hostile_AA Ftr_07 
341.68 70200 99.33 96.71 
256.92 Hostile_AA Ftr_06 
355.94 64.32 O77 32 95.15 
262.99 Hostile_AA Ftr_03/02 
7.96 70 300 70.00 70.00 
264.00 Hostile_SAM_Killer_0l 
1.45 80.00 96.00 96.00 
265.29 Red_AG Ftr_03 
ORs oa! 80.00 80.00 80.00 
268.59 Hostile_AA_Ftr_05 
859.92 70.00 99.88 99.10 
275.52 Red_AG_ Ftr_01/04 
PPA | 70.00 95 750 95.56 
278.50 Hostile_AA_ Ftr_03/01 
13-5 70.00 70.00 70.00 
279.36 Red_AG Ftr_02/04 
Beso 70200 70-00 70', 00 
279.37 Red_CM_04 
312.5% 70°. 00 91.00 91-200 
279.95 Hostile_AA Ftr_02/03 
Ds 25 80.00 94.09 94.09 
280.93 AA_Ftr_03/01 
359.49 45.00 OF 23 FF sacl Ge: 
281.50 Red cM _04 
313.65 70.00 97.30 Saige 
281.76 Hostile _AA Ftr_05 
i. £0 50.00 99.94 99.255 
282.45 Hostile_AA Ftr_02/04 
2258 80.00 94.09 94.09 
262-56. -AA FEr 035/02 
6.41 45.00 45.00 45.00 
282.78 Hostile_AA Ftr_04 
2.04 70.00 99.86 99.63 
283.45 Red_AG_ Ftr_02/03 
BOO? 70.00 I. 52 915.52 
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SSPK Single Shot PK 
PKCum Cumulative PK Of Individual Shots 
Total PK For Killing Target 
Time Target Engager 
iL SSPK PKCum Route 
(Seconds) ID ID 
(Deg) PK 
lSeSsesaeseoaeseSsocent moe Oooo OoO Sooo SeSe CSS Se oSoOoSSeeoSsSes oo OCooOoo Soe See O OCH Om oS Se Se SoHsS 
288.76 AA_Ftr_05/02 Hostile _AA Ftr_01/03 
279.88 A500 6975 69.475 
288.76 AA_Ftr_05/02 Hostile_AA Ftr_01/04 
279.88 45-00 Sees 83.36 
288.85 AA Ftr_05/02 Hostile_AA Ftr_01/01 
281.41 4500 90.85 90.85 
288.88 AA_Ftr_05/02 Hostile_AA Ftr_01/02 
45.00 94.97 94.97 
289.93 AA_Ftr_04/01 Hostile_AA Ftr_03/02 
lea2.6 457.010 99.75 93.74 
293.99 Red_AG Ftr_01/03 AA_Ftr_05/01 
308.40 70-00 92250 92250 
294.00 Hostile_SAM_Killer_01 Asset_Def_SAM_03 
1.86 80.00 99.20 99.20 
294.75 Red_cCM_01 Escort_01/01 
315.80 66.58 99.73 99.73 
294.98 AA_Ftr_03/02 Hostile_AA Ftr_04 
7.39 40.35 67.19 45.08 
295.35 Red CM_01 Escort_01/02 
316.27 65.14 99.91 Doom 
296.63 “AARSrtrE04/ oT Hostile AA Ftr_03/03 
337.79 45.00 99.86 O56 
296.63 AA Ptr04/01 Hostile AA Ftr_03/04 
337.79 45.00 99.92 96.26 
302.31 Hostile_AA Ftr_03/01 AA _ Ftr_04/02 
359.6 70-00 91.00 78.91 
306.72 Red_CM_03 Escort_02/02 
319724 10200 91.00 91.00 
310.89 AA_Ftr_04/01 Hostile AA Ftr_05 
0.28 45.00 99.96 96.26 
Siiees wae Ptr Os702 Hostile_AA Ftr_07 
357249 45.00 SS 95.04 
311.48 Red_cmM 63 Escort_02/01 
322.68 70.00 97.30 SamiBe 
318.42 Hostile_SAM Killer _01 Asset_Def_SAM_03 
98 80.00 99.84 99.84 
319.42 AA_Ftr_04/02 Hostile_AA Ftr_05 
359 257 45.00 94.97 57 264 
322.90 Hostile_AA_Ftr_04 AAS Per 03/701 
0.64 70200 99.96 99.70 
329.11. AALS Fer 04702 Hostile_AA Ftr_03/02 
359.63 45.00 97.23 ba 3i6 
330.06 “AALFtr_047 01 Hostile_AA Ftr_05 
1.68 AS52700 99.98 96.27 
338.26 AA _Ftr_04/02 Hostile_AA Ftr_05 
O10 45.00 98.48 63.43 
336.50 AAS Ptr 703/02 Hostile _AA Ftr_05 
16.37 45200 81.96 45.19 
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215263 


336265 


154.05 


552.62 


211.92 
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247.93 


186.14 
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Scenario: 


Report Type: 


Demo3 00 


PK. HISTORY 


PAPA PK Legend 


Single Shot PK 


PAPA Report 


Report generated on Thu May 


Cumulative PK Of Individual Shots 


PK 
Target 
PKCum Route 
ID 
Ph 
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SSPK 
PKCum 
Route 
Time 
1 SSPK 
(Seconds) 
(Deg) 
We sees cece Ses eee es) em acc 
340.82 
2223 80.00 
350.18 
0.88 45.00 
359.93 
A292 64.01 
360.14 
0205 45.00 
362.09 
0229 70.00 
3672351 
S325 5 ol. 1) 
371.78 
344257 64.90 
372-00 
358.57 70.00 
378.44 
359.79 45.00 
379.60 
356.09 70-00 
382.38 
316.50 62295 
388.24 
355.95 70.00 
393.26 
307.09 70200 
397.38 
0.58 70200 
401.65 
46.50 45.200 
402.26 
310745 70-00 
406.38 
292.47 70.00 
409.42 
355.56 70:00 
4116243 
298.53 70.200 
413.60 
58.35 45.00 
415.29 
5.03 45.00 
415.261 
295.28 70200 


Hostile_AA Ftr_02/03 
98.82 98.82 
AA_Ftr_04/02 
99.16 66.90 
Hostile_AA Ftr_07 
99.76 96.81 
AA_Ftr_04/01 
99.99 96.62 
Hostile_AA Ftr_04 
99.99 99.76 
Hostile_AA Ftr_05 
99.98 99.82 
Hostile_AA Ftr_05 
99.99 99.94 
Hostile_AA Ftr_04 
100.00 99.81 
AA_Ftr_04/02 
99.54 70.04 
Hostile_AA Ftr_04 
100.00 99.84 
Hostile_AA Ftr_05 
1£.0:0:.010 99.98 
Hostile_AA Ftr_03/01 
97.30 79.41 
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99.93 99.04 
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ESsecort_O0l/02 
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ESGort. 02701 

ALO FCr 0470) 

Escorce 02/01 
Hostile_AA Ftr_03/04 
Hostile AA Ftr_03/01 


Escorte01702 
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422.51 Hostile_AA Frr_03/03 Asset_Def_SAM_03 559.04 50.35 
0.09 80.00 Jen 97 90°15 
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428.23 AA _Ftr_03/02 Hostile_AA_ Ftr_04 5267 Slee i 
0.07 45.00 90.08 45223 
430.00 Hostile_AA Ftr_03/04 Escort. 02702 8387 L907 
329-52 70700 96.97 96.97 
Adda) 25 “AACE tr 03701 Hostile _AA Ftr_04 2746 5). aie) 
357.92 45.00 99.16 W2e2e 
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455.20 Red_AG_Ftr_01/03 Asset_Def_SAM_03 39174 61223 
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458.46 AA _Ftr_04/01 Hostile_AA Ftr_03/01 4842 97.24 
207 45.500 FOO 200 97.22 
460.30 Red_AG Ftr_02/03 Escort201701 4187 206.65 
Says 70)'.00 99.93 99.86 
463.00 Red_AG Ftr_02/01 Escort_01/01 2554) 24s 
358.79 70010 70) 20:0 70200 
465.01 AA_Ftr_04/02 Hostile _ AA Ftr_03/03 4498 305.52 
359.80 45.00 99.75 i ea 
468.56 “Escort 02/02 Hostile_AA Ftr_03/04 9297 200.91 


8.69 45.00 45.00 45.00 
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APPENDIX K. PSTAT OUTPUT: DETERMINISTIC TRIALS 


Note: Each page displays data gathered from unwrapped executables above data from wrapped 
executables (C3I, FP, Det). 


Descriptive Statistics 








Variable: Cal 
Cumulative CPU Times 
(unwrapped c3i.pstat) 
Anderson-Darling Normality Test 
A-Squarec: 0.465 
P-Value: 0.236 
Mean 38.5063 
StDev 0.4658 
Variance 0.216996 
Skewness 0.609695 
Kurtosis 5.1E01 
N 30 
Mnimum 37.8800 
1st Quartile 38.1325 
NVedan 38.4750 
3rd Quartile 38.7900 
NMexmum 39.5600 
95% Confidence Interval for Mu 
38.3324 38.6803 
95% Confidence Interval for Sgmna 
0.3710 0.6262 
— 95% Confidence Interval for Median 
5% Confidence Interval for! 
95% Confidence Imtena! for Medan 38 2046 38.6477 
Descriptive Statistics 
P Variable: C2l 
Cumulative CPU Time 
(C3i.pstat) 
Andersor-Darling Normality Test 
A-Squared: 2226 
P-Value: 0.000 
Mean 39.3937 
StDev 0.7044 
Vanance 0.496127 
Skewness 1.07294 
Kurtosis 0.315679 
N 30 
Mnimum 38.1800 
ist Quartile 38.9925 
NVedian 39.1150 
3rd Quartile 39.6700 
Vexmum 41.1200 
95% Confidence Interval for Mu 
39.1307 39.6567 
95% Confidence Interval for Sgqma 
0.5610 0.9469 
95% Confidence Interval for Median 
%, Corfdence internal for Mecan 
95% Corédence interval for Mecar 30.0100 39.3400 
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Descriptive Statistics 








Variable: FP 
Cumulative CPU Times 
(unwrapped fp.pstat) 
Anderson-Darling Normality Test 
A-Squared: 4.232 
P-Value: 0.000 
Mean 24.7327 
StDev 0.9188 
Variance 0.844179 
Skewness 3.87271 
Kurtosis 16.2088 
N 30 
Mnimum 23.8600 
1st Quartile 24.3925 
Median 24.5450 
3rd Quartile 24.8150 
MVexmum 29.2300 
95% Confidence Interval for Mu 
| : : : . : 24.3896 25.0757 
24.35 2445 2455 2465 24.75 24.85 24.9 20S 25.15 95% Confidence Interval for Sigma 
| | | | | | | 
as ae a 
= a eee, . 95% Confidence Interval for Median 
95% Confidence Intenal for Median 24.4200 24.6809 
Descriptive Statistics 
Variable: FP 
Cumulative CPU Time 
(fp.pstat) 
Anderson-Darling Normality Test 
A-Squared: 5.549 
P-Value: 0.000 
Mean 24.8813 
StDev 1.2281 
Vanance 1.50816 
Skewness 429112 
Kurtosis 18.8038 
N 30 
Mnimum 24.2200 
1st Quartile 24.4250 
Median 24.5400 
3rd Quartile 24.8925 
Vexmum 31.0900 





244 24.9 25,4 


5% Confidence Interval for Median 


170 


24.4228 


0.9780 


95% Confidence Interval for Mu 


25.3399 


95% Confidence Interval for Sigma 


1.6509 


95% Confidence Interval for Median 
24.4423 


24.7577 


Descriptive Statistics 





Variable: Detect 
Cumulative CPU Times 
(unwrapped det.pstat) 
Anderson-Darling Normality Test 
_ ASquared: 0.925 
P-Value: 0.016 
Mean 29.2117 
StDev 1.0877 
Variance 1.18299 
Skewness 0.943513 
Kurtosis 1.33429 
N 30 
Mnimum 27.4300 
ist Quartile 28.3475 
Median 29.2400 
3rd Quartile 29.6775 
Vexmum 32.4400 
oN 95% Confidence Interval for Mu 
, 28.8055 29.6178 
= aa = 95% Confidence Interval for Sigma 
| ee er 
: : - a 95% Confidence Intenval for Median 
5% 3 | for Medi 6026 
95°%> Confidence Interve! for Median 28. 29.6471 


Descriptive Statistics 





Variable: Detect 
Cumulative CPU Time 
(detect.pstat) 
Anderson-Darling Normality Test 
A. Squared: 1.014 
P-Value: 0.010 
Mean 38.2990 
StDev 0.3719 
Vanance 0.138306 
Skewness 0.907432 
Kurtosis -1.1E-01 
N 30 
Mnimum 37.7900 
1st Quartile 38.0375 
Median 38.1800 
3rd Quartile 38.5175 
Vaxmum 39.1700 
95% Confidence Interval for Mu 
38.1601 38.4379 
95% Confidence Interval for Sigma 
0.2962 0.4999 
95% Confidence Interval for Medi 
5% Confidence Interval for Median : 48 1014 : eee a 
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AE 
AOR 
ATO 
BMDO 
CZ 

Cs 
C3ISIM 
C4] 

CE 


COMPASS 


COTS 
CPU 
DARPA 
DCA 
DCP 

DIT COE 
DoD 


APPENDIX L: ACRONYMS 


Application Emulator 

Area of Responsibility 

Air Tasking Order 

Ballistic Missile Defense Organization 

Command and Control 

Command, Control, and Communications 

Command, Control, Communications, and Intelligence Simulation 
Command, Control, Communications, Computers, and Intelligence 
Chent Library 

Common Operational Modeling, Planning, and Simulation Strategy 
Commercial Off the Shelf 

Central Processing Unit 

Defense Advanced Research Projects Agency 

Defensive Counter- Air 

Distributed Collaborative Planning Tools 

Defense Information Infrastructure Common Operating Environment 
Department of Defense 

Extended Air Defense Simulation (formerly C3ISIM) 
Executable Editing Library 

Expected Time to Compute 

Fighter Engagement Zone 

Flexible Integrated System Capability 

Flight Processing 

Gigabyte 

Global Command and Control System 
Government Off the Shelf 

Graphic User Interface 

Human Intelligence 

Input and/or Output 

Imagery Intelligence 

Inter Process Communication 

Infrared 

Modeling and Simulation 

Megabyte 

Megabit per Second 

MSHN Daemon 

Missile Engagement Zone 

Measure of Effectiveness 

Megahertz 

Millions of Instructions Per Second 

Multiple Rocket System 

Management System for Heterogeneous Networks 
National Command Authority 

Network File System 

Network File Service 

Numerical Aerodynamic Simulation 

Object Management Group 

Object Request Broker 

Probability of Kall 

Protocol Stack Instance Identifier 

Protocol System Protocol 

Quality of Service 

Random Access Memory 


ie, 


SPAWAR 
TBE 
TBM 
TMD 


Read Only Memory 

Resource Management System 
Remote Procedure Call 

Resource Requirements Database 
Resource Status Server 

Scheduling Advisor 

Surface to Air Missile 

Suppression of Enemy Air Defense 
Signals Intelligence 

Space and Missile Defense Command 
Space and Naval Warfare 
Teledyne Brown Engineering 
Tactical Ballistic Missile 

Theater Missile Defense 
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